diff --git a/doc/PADDLE_SERVING_ON_KUBERNETES.md b/doc/PADDLE_SERVING_ON_KUBERNETES.md
index 399cf156dc4b890c02e8da5bec5aa132663d7d2d..943b82042cd2c1908fe19bc0c015c0cb375664ef 100644
--- a/doc/PADDLE_SERVING_ON_KUBERNETES.md
+++ b/doc/PADDLE_SERVING_ON_KUBERNETES.md
@@ -4,9 +4,9 @@ Paddle Serving在0.6.0版本开始支持在Kubenetes集群上部署,并提供
### 集群准备
-如果您还没有Kubenetes集群,我们推荐[购买并使用百度智能云CCE集群](). 如果是其他云服务商提供的集群,或者自行安装Kubenetes集群,请遵照对应的教程。
+如果您还没有Kubenetes集群,我们推荐[购买并使用百度智能云CCE集群](https://cloud.baidu.com/doc/CCE/index.html). 如果是其他云服务商提供的集群,或者自行安装Kubenetes集群,请遵照对应的教程。
-您还需要准备一个用于Kubenetes集群部署使用的镜像仓库,通常与云服务提供商绑定,如果您使用的是百度智能云的CCE集群,可以参照[百度智能云CCR镜像仓库使用方式]()。当然Docker Hub也可以作为镜像仓库,但是可能在部署时会出现下载速度慢的情况。
+您还需要准备一个用于Kubenetes集群部署使用的镜像仓库,通常与云服务提供商绑定,如果您使用的是百度智能云的CCE集群,可以参照[百度智能云CCR镜像仓库使用方式](https://cloud.baidu.com/doc/CCR/index.html)。当然Docker Hub也可以作为镜像仓库,但是可能在部署时会出现下载速度慢的情况。
### 环境准备
diff --git a/doc/SERVING_AUTH_DOCKER.md b/doc/SERVING_AUTH_DOCKER.md
new file mode 100644
index 0000000000000000000000000000000000000000..2ef2bbb8ab9341e7c3aca55dc63b3f7bd80b54ca
--- /dev/null
+++ b/doc/SERVING_AUTH_DOCKER.md
@@ -0,0 +1,199 @@
+# 在Paddle Serving使用安全网关
+
+## 简介
+
+在之前的服务部署示例中,我们都从开发的角度切入,然而,在现实的生产环境中,仅仅提供一个能够预测的远端服务接口还远远不够。我们仍然要考虑以下不足。
+
+- 这个服务还不能以网关的形式提供,访问路径难以管理。
+- 这个服务接口不够安全,需要做相应的鉴权。
+- 这个服务接口不能够控制流量,无法合理利用资源。
+
+本文档的作用,就以 Uci 房价预测服务为例,来介绍如何强化预测服务API接口安全。API网关作为流量入口,对接口进行统一管理。但API网关可以提供流量加密和鉴权等安全功能。
+
+## Docker部署
+
+可以使用docker-compose来部署安全网关。这个示例的步骤就是 [部署本地Serving容器] - [部署本地安全网关] - [通过安全网关访问Serving]
+
+**注明:** docker-compose与docker不一样,它依赖于docker,一次可以部署多个docker容器,可以类比于本地版的kubenetes,docker-compose的教程请参考[docker-compose安装](https://docs.docker.com/compose/install/)
+
+```shell
+docker-compose -f tools/auth/auth-serving-docker.yaml up -d
+```
+
+可以通过 `docker ps` 来查看启动的容器。
+
+```shell
+3035cf445029 pantsel/konga:next "/app/start.sh" About an hour ago Up About an hour 0.0.0.0:8005->1337/tcp anquan_konga_1
+7ce3abee550c registry.baidubce.com/serving_gateway/kong:paddle "/docker-entrypoint.…" About an hour ago Up About an hour (healthy) 0.0.0.0:8000->8000/tcp, 127.0.0.1:8001->8001/tcp, 0.0.0.0:8443->8443/tcp, 127.0.0.1:8444->8444/tcp anquan_kong_1
+25810fd79a27 postgres:9.6 "docker-entrypoint.s…" About an hour ago Up About an hour (healthy) 5432/tcp anquan_db_1
+ee59a3dd4806 registry.baidubce.com/serving_dev/serving-runtime:cpu-py36 "bash -c ' wget --no…" About an hour ago Up About an hour 0.0.0.0:9393->9393/tcp anquan_serving_1
+665fd8a34e15 redis:latest "docker-entrypoint.s…" About an hour ago Up About an hour 0.0.0.0:6379->6379/tcp anquan_redis_1
+```
+
+其中我们之前serving容器 以 9393端口暴露,KONG网关的端口是8443, KONG的Web控制台的端口是8001。接下来我们在浏览器访问 `https://$IP_ADDR:8001`, 其中 IP_ADDR就是宿主机的IP。
+
+
+可以看到在注册结束后,登陆,看到了 DASHBOARD,我们先看SERVICES,可以看到`serving_service`,这意味着我们端口在9393的Serving服务已经在KONG当中被注册。
+
+
+
+
+然后在ROUTES中,我们可以看到 serving 被链接到了 `/serving-uci`。
+
+最后我们点击 CONSUMERS - default_user - Credentials - API KEYS ,我们可以看到 `Api Keys` 下看到很多key
+
+
+
+接下来可以通过curl访问
+
+```shell
+ curl -H "Content-Type:application/json" -H "X-INSTANCE-ID:kong_ins" -H "apikey:hP6v25BQVS5CcS1nqKpxdrFkUxze9JWD" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' https://127.0.0.1:8443/serving-uci/uci/prediction -k
+```
+
+与之前的Serving HTTP服务相比,有以下区别。
+
+- 使用https加密访问,而不是http
+- 使用serving_uci的路径映射到网关
+- 在header处增加了 `X-INSTANCE-ID`和`apikey`
+
+
+## K8S部署
+
+同样,我们也提供了K8S集群部署Serving安全网关的方式。
+
+### Step 1:启动Serving服务
+
+我们仍然以 [Uci房价预测](../python/examples/fit_a_line)服务作为例子,这里省略了镜像制作的过程,详情可以参考 [在Kubernetes集群上部署Paddle Serving](./PADDLE_SERVING_ON_KUBERNETES.md)。
+
+在这里我们直接执行
+```
+kubectl apply -f tools/auth/serving-demo-k8s.yaml
+```
+
+可以看到
+
+### Step 2: 安装 KONG (一个集群只需要执行一次就可以)
+接下来我们执行KONG Ingress的安装
+```
+kubectl apply -f tools/auth/kong-install.yaml
+```
+
+输出是
+```
+namespace/kong created
+customresourcedefinition.apiextensions.k8s.io/kongclusterplugins.configuration.konghq.com created
+customresourcedefinition.apiextensions.k8s.io/kongconsumers.configuration.konghq.com created
+customresourcedefinition.apiextensions.k8s.io/kongingresses.configuration.konghq.com created
+customresourcedefinition.apiextensions.k8s.io/kongplugins.configuration.konghq.com created
+customresourcedefinition.apiextensions.k8s.io/tcpingresses.configuration.konghq.com created
+serviceaccount/kong-serviceaccount created
+clusterrole.rbac.authorization.k8s.io/kong-ingress-clusterrole created
+clusterrolebinding.rbac.authorization.k8s.io/kong-ingress-clusterrole-nisa-binding created
+service/kong-proxy created
+service/kong-validation-webhook created
+deployment.apps/ingress-kong created
+```
+我们可以输入
+```
+kubectl get service --all-namespaces
+```
+会显示
+```
+NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
+default uci ClusterIP 172.16.87.89 9393/TCP 7d7h
+kong kong-proxy NodePort 172.16.23.91 80:8175/TCP,443:8521/TCP 102m
+kong kong-validation-webhook ClusterIP 172.16.114.93 443/TCP 102m
+
+```
+
+### Step 3: 创建Ingress资源
+
+接下来需要做Serving服务和KONG的链接
+
+```
+kubectl apply -f tools/auth/kong-ingress-k8s.yaml
+```
+
+我们也给出yaml文件内容
+```
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+ name: demo
+ annotations:
+ konghq.com/strip-path: "true"
+ kubernetes.io/ingress.class: kong
+spec:
+ rules:
+ - http:
+ paths:
+ - path: /foo
+ backend:
+ serviceName: {{SERVING_SERVICE_NAME}}
+ servicePort: {{SERVICE_PORT}}
+```
+其中serviceName就是uci,servicePort就是9393,如果是别的服务就需要改这两个字段,最终会映射到`/foo`下。
+在这一步之后,我们就可以
+```
+curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' http://$IP:$PORT/foo/uci/prediction
+```
+
+### Step 4: 增加安全网关限制
+
+之前的接口没有鉴权功能,无法验证用户身份合法性,现在我们添加一个key-auth插件
+
+执行
+```
+kubectl apply -f key-auth-k8s.yaml
+```
+
+其中,yaml文内容为
+```
+apiVersion: configuration.konghq.com/v1
+kind: KongPlugin
+metadata:
+ name: key-auth
+plugin: key-auth
+```
+
+现在,需要创建secret,key值为用户指定,需要在请求时携带Header中apikey字段
+执行
+```
+kubectl create secret generic default-apikey \
+ --from-literal=kongCredType=key-auth \
+ --from-literal=key=ZGVmYXVsdC1hcGlrZXkK
+```
+
+在这里,我们的key是随意制定了一串 `ZGVmYXVsdC1hcGlrZXkK`,实际情况也可以
+创建一个用户(consumer)标识访问者身份,并未该用户绑定apikey。
+执行
+```
+kubectl apply -f kong-consumer-k8s.yaml
+```
+
+其中,yaml文内容为
+```
+apiVersion: configuration.konghq.com/v1
+kind: KongConsumer
+metadata:
+ name: default
+ annotations:
+ kubernetes.io/ingress.class: kong
+username: default
+credentials:
+- default-apikey
+```
+
+如果我们这时还想再像上一步一样的做curl访问,会发现已经无法访问,此时已经具备了安全能力,我们需要对应的key。
+
+
+### Step 5: 通过API Key访问服务
+
+执行
+```
+curl -H "Content-Type:application/json" -H "apikey:ZGVmYXVsdC1hcGlrZXkK" -X POST -d '{"feed":[{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]}], "fetch":["price"]}' https://$IP:$PORT/foo/uci/prediction -k
+```
+我们可以看到 apikey 已经加入到了curl请求的header当中。
+
+
+
diff --git a/doc/kong-api_keys.png b/doc/kong-api_keys.png
new file mode 100644
index 0000000000000000000000000000000000000000..0b0413315a3940f8dde83c618a8563b683f73995
Binary files /dev/null and b/doc/kong-api_keys.png differ
diff --git a/doc/kong-dashboard.png b/doc/kong-dashboard.png
new file mode 100644
index 0000000000000000000000000000000000000000..f5f383a24df0aa83fc3fbf61bb85ae1dc7adada0
Binary files /dev/null and b/doc/kong-dashboard.png differ
diff --git a/doc/kong-routes.png b/doc/kong-routes.png
new file mode 100644
index 0000000000000000000000000000000000000000..541f89a5dbc7d030db64f795c42cea1bfc85ce45
Binary files /dev/null and b/doc/kong-routes.png differ
diff --git a/doc/kong-services.png b/doc/kong-services.png
new file mode 100644
index 0000000000000000000000000000000000000000..e7e1e46d8f91b2f8718e3895e93741cb42598cdd
Binary files /dev/null and b/doc/kong-services.png differ
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/README.md b/python/examples/pipeline/PaddleClas/DarkNet53/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/README_CN.md b/python/examples/pipeline/PaddleClas/DarkNet53/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.py b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.sh b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..ee20eccf64c1e08dff16c7427ae5107525281dee
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-DarkNet53"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..273d687d7a43c0bec5b06c7fd4484e156b0aeddc
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "DarkNet53"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/DarkNet53_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/config.yml b/python/examples/pipeline/PaddleClas/DarkNet53/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..58eb80c2c6dc08e16c81667bdbea55e2f31dc8d7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: DarkNet53/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/cpu_utilization.py b/python/examples/pipeline/PaddleClas/DarkNet53/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/daisy.jpg b/python/examples/pipeline/PaddleClas/DarkNet53/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/DarkNet53/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/get_model.sh b/python/examples/pipeline/PaddleClas/DarkNet53/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..b19bd02d24ad82188a788c0a825616d21b6807b8
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/DarkNet53.tar
+tar -xf DarkNet53.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/imagenet.label b/python/examples/pipeline/PaddleClas/DarkNet53/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/DarkNet53/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/DarkNet53/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..c16966e3c4d9481b03344bc51abcd2a2090e5bb7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/DarkNet53/resnet50_web_service.py
@@ -0,0 +1,71 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/README.md b/python/examples/pipeline/PaddleClas/HRNet_W18_C/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/README_CN.md b/python/examples/pipeline/PaddleClas/HRNet_W18_C/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.py b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.sh b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..9abbd0b362e6b85dc17ac29dd60eac8b88bbbb18
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-HRNet_W18_C"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..d18be38d776a0f46aced2e550c70c02c1ac8fb12
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "HRNet_W18_C"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/HRNet_W18_C_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/config.yml b/python/examples/pipeline/PaddleClas/HRNet_W18_C/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..c02ed38c2df8d5d623839e3d1dd5df274f95e96a
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: HRNet_W18_C/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/cpu_utilization.py b/python/examples/pipeline/PaddleClas/HRNet_W18_C/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/daisy.jpg b/python/examples/pipeline/PaddleClas/HRNet_W18_C/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/HRNet_W18_C/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/get_model.sh b/python/examples/pipeline/PaddleClas/HRNet_W18_C/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..73381edb1b09693150867680da9031c7cbf081ec
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/HRNet_W18_C.tar
+tar -xf HRNet_W18_C.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/imagenet.label b/python/examples/pipeline/PaddleClas/HRNet_W18_C/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/HRNet_W18_C/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/HRNet_W18_C/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/HRNet_W18_C/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/README.md b/python/examples/pipeline/PaddleClas/MobileNetV1/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/README_CN.md b/python/examples/pipeline/PaddleClas/MobileNetV1/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.py b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.sh b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..d10142449db8b2ad3ea39e6e7531158b833b438a
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-MobileNetV1"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..86d84934b99649981998cee10f72a3864f28716e
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "MobileNetV1"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV1_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/config.yml b/python/examples/pipeline/PaddleClas/MobileNetV1/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..6be4d209d7fba90ae6f1b0e9e30f044bc0d6dd45
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: MobileNetV1/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/cpu_utilization.py b/python/examples/pipeline/PaddleClas/MobileNetV1/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/daisy.jpg b/python/examples/pipeline/PaddleClas/MobileNetV1/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/MobileNetV1/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/get_model.sh b/python/examples/pipeline/PaddleClas/MobileNetV1/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..3cc56d3a4edb2d56c1fbf80c023c2eaefffd6dca
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/MobileNetV1.tar
+tar -xf MobileNetV1.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/imagenet.label b/python/examples/pipeline/PaddleClas/MobileNetV1/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV1/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/MobileNetV1/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV1/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/README.md b/python/examples/pipeline/PaddleClas/MobileNetV2/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/README_CN.md b/python/examples/pipeline/PaddleClas/MobileNetV2/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.py b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.sh b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..53ede7915316ef23c1208c02b624ad4a327733e6
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-MobileNetV2"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..7f3ac0790fef2604529dd0107c863d7346ffa077
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "MobileNetV2"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV2_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..872dee1f5ee7d568fca8337897bf3232fcabcae1
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/config.yml b/python/examples/pipeline/PaddleClas/MobileNetV2/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..2b229673a8800e5ef2d6c5a5b9c1c107fb2a1041
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: MobileNetV2/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/cpu_utilization.py b/python/examples/pipeline/PaddleClas/MobileNetV2/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/daisy.jpg b/python/examples/pipeline/PaddleClas/MobileNetV2/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/MobileNetV2/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/get_model.sh b/python/examples/pipeline/PaddleClas/MobileNetV2/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..e3262f264a2859ef508581e46e54eef3d8732684
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/MobileNetV2.tar
+tar -xf MobileNetV2.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/imagenet.label b/python/examples/pipeline/PaddleClas/MobileNetV2/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV2/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/MobileNetV2/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV2/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README.md b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README_CN.md b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.py b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.sh b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..6a431f10ac7bdd05266e59318caafae4c223f427
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-MobileNetV3_large_x1_0"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..1d9c39d3e078cbd92cd2f9f82ba053e1bce8f79d
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "MobileNetV3_large_x1_0"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/MobileNetV3_large_x1_0_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..bf7fea1efe78666059b7ff2a0dd51bb6991ee5fa
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/benchmark_gpu.sh
@@ -0,0 +1,41 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/config.yml b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..c27cccc82278d80bbadd36c1494400c2b4bb249d
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 2
+
+ #uci模型路径
+ model_config: MobileNetV3_large_x1_0/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/cpu_utilization.py b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/daisy.jpg b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/get_model.sh b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..d2135801fb5b804e37248f17a579d095f8e0afb3
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/MobileNetV3_large_x1_0.tar
+tar -xf MobileNetV3_large_x1_0.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/imagenet.label b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/MobileNetV3_large_x1_0/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README.md b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README_CN.md b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.py b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..2b9ec90e65a2c58e1b4b6b5c0237c6007e49c71e
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNeXt101_vd_64x4d"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..cd73b0f10c3008808787a14006ca8fe87a85e697
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ResNeXt101_vd_64x4d"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ResNeXt101_vd_64x4d_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/config.yml b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..df7bb45264a43a6816d081e567b5ee7d9366b54c
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ResNeXt101_vd_64x4d/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/get_model.sh b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..603aa286134516c4956f1078e05fda5a84d57ca1
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ResNeXt101_vd_64x4d.tar
+tar -xf ResNeXt101_vd_64x4d.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/imagenet.label b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNeXt101_vd_64x4d/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/README.md b/python/examples/pipeline/PaddleClas/ResNet50_vd/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/README_CN.md b/python/examples/pipeline/PaddleClas/ResNet50_vd/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.py b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..bc1c65244658912f6c700e3c342cec56813de4d3
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNet50_vd"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..085d9f9d33d6670bcfcd8b9b113eb5413f7a7046
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ResNet50_vd"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/config.yml b/python/examples/pipeline/PaddleClas/ResNet50_vd/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..f946dc3764a9c84e8efc3df594e4013d56b80c91
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ResNet50_vd/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ResNet50_vd/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNet50_vd/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNet50_vd/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/get_model.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..7b4f0b243542d5c0c5cb81e50bb4b423272c730d
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ResNet50_vd.tar
+tar -xf ResNet50_vd.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/imagenet.label b/python/examples/pipeline/PaddleClas/ResNet50_vd/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNet50_vd/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README_CN.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..17593214deccfa2f1af070ebd71c82775273019c
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNet50_vd_FPGM"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..4338b381f3df360fa71e7039ca1636b55262c45f
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ResNet50_vd_FPGM"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_FPGM_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/config.yml b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..cc40ee56df90f674fc9fcce2affc4fb06e2f16a1
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ResNet50_vd_FPGM/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["save_infer_model/scale_0.tmp_1"]
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/get_model.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..4e1414e92c19489249414ae0904509796bfe99b6
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ResNet50_vd_FPGM.tar
+tar -xf ResNet50_vd_FPGM.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/imagenet.label b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..668b119273df6ab351e5234badb98b41bef87c1e
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_FPGM/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["save_infer_model/scale_0.tmp_1"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README_CN.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..248f9506e346aa15fd347f91f811e95657fdb66b
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNet50_vd_KL"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..786c5d11b3764276fceaff4c87cdcb28f05688b8
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ResNet50_vd_KL"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_KL_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/config.yml b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..b728e4e2d81d9bbc5ba2c41d7c10ae364c78178d
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ResNet50_vd_KL/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["save_infer_model/scale_0.tmp_0"]
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/get_model.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..d81b8eb53b0317c364fcff5b9b47f9a7d6075e55
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ResNet50_vd_KL.tar
+tar -xf ResNet50_vd_KL.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/imagenet.label b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..359ea1ed817c7117f64e68fd8a984aa0e7bf5f60
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_KL/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"inputs": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["save_infer_model/scale_0.tmp_0"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README_CN.md b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..06cbe71821afe257202c5088880d467053d6d762
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNet50_vd_PACT"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..4387732519dfc033a29dd3f797e66ec97a5200a4
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ResNet50_vd_PACT"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ResNet50_vd_PACT_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/config.yml b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..b97581662a49bd04da6b73b11720ac1ebbee07db
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ResNet50_vd_PACT/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["save_infer_model/scale_0.tmp_1"]
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/get_model.sh b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..d02a90c6b5aa97e757623582f5640ee923bfb6e8
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ResNet50_vd_PACT.tar
+tar -xf ResNet50_vd_PACT.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/imagenet.label b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..2c35ab72255fe2fabd9c83d7a3bd152b744bdd8e
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet50_vd_PACT/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"inputs": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["save_infer_model/scale_0.tmp_1"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/README.md b/python/examples/pipeline/PaddleClas/ResNet_V2_50/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..5b909301d9e114019ae8c6ac2bbfcec3cb188b33
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/README.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+python -m paddle_serving_app.package --get_model resnet_v2_50_imagenet
+tar -xzvf resnet_v2_50_imagenet.tar.gz
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/README_CN.md b/python/examples/pipeline/PaddleClas/ResNet_V2_50/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..cc2fcdd7514fc197ec892826ec56b76906150578
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/README_CN.md
@@ -0,0 +1,21 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+python -m paddle_serving_app.package --get_model resnet_v2_50_imagenet
+tar -xzvf resnet_v2_50_imagenet.tar.gz
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.py b/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..98babc4acddb9a548afeafed1dfee16a88244714
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18000/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.sh b/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..14c82dfcca801bc00bec57ef972f2260dd1d844a
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ResNet_v2_50"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/config.yml b/python/examples/pipeline/PaddleClas/ResNet_V2_50/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..d6a1d44af70e1a72dd24461ae2d4bc575a97a160
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: resnet_v2_50_imagenet_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["score"]
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/daisy.jpg b/python/examples/pipeline/PaddleClas/ResNet_V2_50/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ResNet_V2_50/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ResNet_V2_50/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..28f2d1c618b0bcaf296b5365e679a2b45784624f
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18000/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(1):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ResNet_V2_50/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ResNet_V2_50/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..8b3815f7d7d1397ffd7618048a43a21b5b3123e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ResNet_V2_50/resnet50_web_service.py
@@ -0,0 +1,71 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["score"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README.md b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..d0fa99e6d72f10d3d2b5907285528b68685128e0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README.md
@@ -0,0 +1,19 @@
+# Imagenet Pipeline WebService
+
+This document will takes Imagenet service as an example to introduce how to use Pipeline WebService.
+
+## Get model
+```
+sh get_model.sh
+```
+
+## Start server
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## RPC test
+```
+python pipeline_rpc_client.py
+```
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README_CN.md b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README_CN.md
new file mode 100644
index 0000000000000000000000000000000000000000..335c96b2144b17e20d6007f376dec4416fb10aa5
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/README_CN.md
@@ -0,0 +1,20 @@
+# Imagenet Pipeline WebService
+
+这里以 Imagenet 服务为例来介绍 Pipeline WebService 的使用。
+
+## 获取模型
+```
+sh get_model.sh
+```
+
+## 启动服务
+
+```
+python resnet50_web_service.py &>log.txt &
+```
+
+## 测试
+```
+python pipeline_rpc_client.py
+```
+
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.py b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.py
new file mode 100644
index 0000000000000000000000000000000000000000..2433b0132728dc96627254f9231949a74a551c28
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.py
@@ -0,0 +1,134 @@
+import sys
+import os
+import base64
+import yaml
+import requests
+import time
+import json
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+from paddle_serving_client.utils import MultiThreadRunner
+from paddle_serving_client.utils import benchmark_args, show_latency
+def parse_benchmark(filein, fileout):
+ with open(filein, "r") as fin:
+ res = yaml.load(fin)
+ del_list = []
+ for key in res["DAG"].keys():
+ if "call" in key:
+ del_list.append(key)
+ for key in del_list:
+ del res["DAG"][key]
+ with open(fileout, "w") as fout:
+ yaml.dump(res, fout, default_flow_style=False)
+
+def gen_yml(device, gpu_id):
+ fin = open("config.yml", "r")
+ config = yaml.load(fin)
+ fin.close()
+ config["dag"]["tracer"] = {"interval_s": 10}
+ if device == "gpu":
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 1
+ config["op"]["imagenet"]["local_service_conf"]["devices"] = gpu_id
+ else:
+ config["op"]["imagenet"]["local_service_conf"]["device_type"] = 0
+ with open("config2.yml", "w") as fout:
+ yaml.dump(config, fout, default_flow_style=False)
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+def run_http(idx, batch_size):
+ print("start thread ({})".format(idx))
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ start = time.time()
+
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ keys, values = [], []
+ for i in range(batch_size):
+ keys.append("image_{}".format(i))
+ values.append(image)
+ data = {"key": keys, "value": values}
+ latency_list = []
+ start_time = time.time()
+ total_num = 0
+ while True:
+ l_start = time.time()
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
+ l_end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
+ total_num += 1
+ if time.time() - start_time > 20:
+ break
+ end = time.time()
+ return [[end - start], latency_list, [total_num]]
+
+def multithread_http(thread, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
+
+def run_rpc(thread, batch_size):
+ client = PipelineClient()
+ client.connect(['127.0.0.1:18080'])
+ start = time.time()
+ test_img_dir = "imgs/"
+ for img_file in os.listdir(test_img_dir):
+ with open(os.path.join(test_img_dir, img_file), 'rb') as file:
+ image_data = file.read()
+ image = cv2_to_base64(image_data)
+ start_time = time.time()
+ while True:
+ ret = client.predict(feed_dict={"image": image}, fetch=["res"])
+ if time.time() - start_time > 10:
+ break
+ end = time.time()
+ return [[end - start]]
+
+
+def multithread_rpc(thraed, batch_size):
+ multi_thread_runner = MultiThreadRunner()
+ result = multi_thread_runner.run(run_rpc , thread, batch_size)
+
+if __name__ == "__main__":
+ if sys.argv[1] == "yaml":
+ mode = sys.argv[2] # brpc/ local predictor
+ thread = int(sys.argv[3])
+ device = sys.argv[4]
+ if device == "gpu":
+ gpu_id = sys.argv[5]
+ else:
+ gpu_id = None
+ gen_yml(device, gpu_id)
+ elif sys.argv[1] == "run":
+ mode = sys.argv[2] # http/ rpc
+ thread = int(sys.argv[3])
+ batch_size = int(sys.argv[4])
+ if mode == "http":
+ multithread_http(thread, batch_size)
+ elif mode == "rpc":
+ multithread_rpc(thread, batch_size)
+ elif sys.argv[1] == "dump":
+ filein = sys.argv[2]
+ fileout = sys.argv[3]
+ parse_benchmark(filein, fileout)
+
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.sh b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.sh
new file mode 100644
index 0000000000000000000000000000000000000000..08523c4a5ef929ffb5ce9c30f3c1059799a4f9f0
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark.sh
@@ -0,0 +1,44 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.6"
+modelname="clas-ShuffleNetV2_x1_0"
+
+# HTTP
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
+rm -rf profile_log_$modelname
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
+do
+ for batch_size in 1
+ do
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
+ done
+done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..d7b853688c7c4161a9a5c2699c6fe14dc9bf0c57
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "ShuffleNetV2_x1_0"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "https://paddle-imagenet-models-name.bj.bcebos.com/ShuffleNetV2_x1_0_pretrained.tar"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "gpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml.template b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml.template
new file mode 100644
index 0000000000000000000000000000000000000000..e98ffb74a35533d831999b6e47bb1acafb0648ff
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_config.yaml.template
@@ -0,0 +1,32 @@
+
+cuda_version: "10.1"
+cudnn_version: "7.6"
+trt_version: "6.0"
+python_version: "3.7"
+gcc_version: "8.2"
+paddle_version: "2.0.1"
+
+cpu: "Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz X12"
+gpu: "T4"
+xpu: "None"
+api: ""
+owner: "cuicheng01"
+
+model_name: "imagenet"
+model_type: "static"
+model_source: "PaddleClas"
+model_url: "model_url_path"
+
+batch_size: 1
+num_of_samples: 1000
+input_shape: "3,224,224"
+
+runtime_device: "cpu"
+ir_optim: true
+enable_memory_optim: true
+enable_tensorrt: false
+precision: "fp32"
+enable_mkldnn: false
+cpu_math_library_num_threads: ""
+
+
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_gpu.sh b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..8b249ca5d344a140ca7165f531bd63be0bfade61
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/benchmark_gpu.sh
@@ -0,0 +1,42 @@
+export FLAGS_profile_pipeline=1
+alias python3="python3.7"
+modelname="imagenet"
+use_gpu=1
+gpu_id="0"
+benchmark_config_filename="benchmark_config.yaml"
+
+# HTTP
+ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+sleep 3
+if [ $use_gpu -eq 1 ]; then
+ python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+else
+ python3 benchmark.py yaml local_predictor 1 cpu
+fi
+rm -rf profile_log_$modelname
+for thread_num in 1
+do
+ for batch_size in 1
+ do
+ echo "#----imagenet thread num: $thread_num batch size: $batch_size mode:http use_gpu:$use_gpu----" >>profile_log_$modelname
+ rm -rf PipelineServingLogs
+ rm -rf cpu_utilization.py
+ python3 resnet50_web_service.py >web.log 2>&1 &
+ sleep 3
+ nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
+ nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
+ python3 benchmark.py run http $thread_num $batch_size
+ python3 cpu_utilization.py >>profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
+ ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+ ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
+ python3 benchmark.py dump benchmark.log benchmark.tmp
+ mv benchmark.tmp benchmark.log
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
+ cat benchmark.log >> profile_log_$modelname
+ python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
+ #rm -rf gpu_use.log gpu_utilization.log
+ done
+done
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/config.yml b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/config.yml
new file mode 100644
index 0000000000000000000000000000000000000000..eeea272e66a6aad5a9899f6e4624d2e058fd294f
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/config.yml
@@ -0,0 +1,33 @@
+#worker_num, 最大并发数。当build_dag_each_worker=True时, 框架会创建worker_num个进程,每个进程内构建grpcSever和DAG
+##当build_dag_each_worker=False时,框架会设置主线程grpc线程池的max_workers=worker_num
+worker_num: 1
+
+#http端口, rpc_port和http_port不允许同时为空。当rpc_port可用且http_port为空时,不自动生成http_port
+http_port: 18080
+rpc_port: 9993
+
+dag:
+ #op资源类型, True, 为线程模型;False,为进程模型
+ is_thread_op: False
+op:
+ imagenet:
+ #当op配置没有server_endpoints时,从local_service_conf读取本地服务配置
+ local_service_conf:
+
+ #并发数,is_thread_op=True时,为线程并发;否则为进程并发
+ concurrency: 1
+
+ #uci模型路径
+ model_config: ShuffleNetV2_x1_0/ppcls_model/
+
+ #计算硬件类型: 空缺时由devices决定(CPU/GPU),0=cpu, 1=gpu, 2=tensorRT, 3=arm cpu, 4=kunlun xpu
+ device_type: 1
+
+ #计算硬件ID,当devices为""或不写时为CPU预测;当devices为"0", "0,1,2"时为GPU预测,表示使用的GPU卡
+ devices: "0" # "0,1"
+
+ #client类型,包括brpc, grpc和local_predictor.local_predictor不启动Serving服务,进程内预测
+ client_type: local_predictor
+
+ #Fetch结果列表,以client_config中fetch_var的alias_name为准
+ fetch_list: ["prediction"]
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/cpu_utilization.py b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/cpu_utilization.py
new file mode 100644
index 0000000000000000000000000000000000000000..984c72370a3551722b21b8e06d418efd832192ef
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/cpu_utilization.py
@@ -0,0 +1,4 @@
+import psutil
+cpu_utilization=psutil.cpu_percent(1,False)
+print('CPU_UTILIZATION:', cpu_utilization)
+
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/daisy.jpg b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/daisy.jpg
new file mode 100644
index 0000000000000000000000000000000000000000..7edeca63e5f32e68550ef720d81f59df58a8eabc
Binary files /dev/null and b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/daisy.jpg differ
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/get_model.sh b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/get_model.sh
new file mode 100644
index 0000000000000000000000000000000000000000..6a0611ce6145454db1c90c5a798d610c5b2d6888
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/get_model.sh
@@ -0,0 +1,5 @@
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/model/ShuffleNetV2_x1_0.tar
+tar -xf ShuffleNetV2_x1_0.tar
+
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/imagenet-example/image_data.tar.gz
+tar -xzvf image_data.tar.gz
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/imagenet.label b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/imagenet.label
new file mode 100644
index 0000000000000000000000000000000000000000..d7146735146ea1894173d6d0e20fb90af36be849
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/imagenet.label
@@ -0,0 +1,1000 @@
+tench, Tinca tinca,
+goldfish, Carassius auratus,
+great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias,
+tiger shark, Galeocerdo cuvieri,
+hammerhead, hammerhead shark,
+electric ray, crampfish, numbfish, torpedo,
+stingray,
+cock,
+hen,
+ostrich, Struthio camelus,
+brambling, Fringilla montifringilla,
+goldfinch, Carduelis carduelis,
+house finch, linnet, Carpodacus mexicanus,
+junco, snowbird,
+indigo bunting, indigo finch, indigo bird, Passerina cyanea,
+robin, American robin, Turdus migratorius,
+bulbul,
+jay,
+magpie,
+chickadee,
+water ouzel, dipper,
+kite,
+bald eagle, American eagle, Haliaeetus leucocephalus,
+vulture,
+great grey owl, great gray owl, Strix nebulosa,
+European fire salamander, Salamandra salamandra,
+common newt, Triturus vulgaris,
+eft,
+spotted salamander, Ambystoma maculatum,
+axolotl, mud puppy, Ambystoma mexicanum,
+bullfrog, Rana catesbeiana,
+tree frog, tree-frog,
+tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui,
+loggerhead, loggerhead turtle, Caretta caretta,
+leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea,
+mud turtle,
+terrapin,
+box turtle, box tortoise,
+banded gecko,
+common iguana, iguana, Iguana iguana,
+American chameleon, anole, Anolis carolinensis,
+whiptail, whiptail lizard,
+agama,
+frilled lizard, Chlamydosaurus kingi,
+alligator lizard,
+Gila monster, Heloderma suspectum,
+green lizard, Lacerta viridis,
+African chameleon, Chamaeleo chamaeleon,
+Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis,
+African crocodile, Nile crocodile, Crocodylus niloticus,
+American alligator, Alligator mississipiensis,
+triceratops,
+thunder snake, worm snake, Carphophis amoenus,
+ringneck snake, ring-necked snake, ring snake,
+hognose snake, puff adder, sand viper,
+green snake, grass snake,
+king snake, kingsnake,
+garter snake, grass snake,
+water snake,
+vine snake,
+night snake, Hypsiglena torquata,
+boa constrictor, Constrictor constrictor,
+rock python, rock snake, Python sebae,
+Indian cobra, Naja naja,
+green mamba,
+sea snake,
+horned viper, cerastes, sand viper, horned asp, Cerastes cornutus,
+diamondback, diamondback rattlesnake, Crotalus adamanteus,
+sidewinder, horned rattlesnake, Crotalus cerastes,
+trilobite,
+harvestman, daddy longlegs, Phalangium opilio,
+scorpion,
+black and gold garden spider, Argiope aurantia,
+barn spider, Araneus cavaticus,
+garden spider, Aranea diademata,
+black widow, Latrodectus mactans,
+tarantula,
+wolf spider, hunting spider,
+tick,
+centipede,
+black grouse,
+ptarmigan,
+ruffed grouse, partridge, Bonasa umbellus,
+prairie chicken, prairie grouse, prairie fowl,
+peacock,
+quail,
+partridge,
+African grey, African gray, Psittacus erithacus,
+macaw,
+sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita,
+lorikeet,
+coucal,
+bee eater,
+hornbill,
+hummingbird,
+jacamar,
+toucan,
+drake,
+red-breasted merganser, Mergus serrator,
+goose,
+black swan, Cygnus atratus,
+tusker,
+echidna, spiny anteater, anteater,
+platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus,
+wallaby, brush kangaroo,
+koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus,
+wombat,
+jellyfish,
+sea anemone, anemone,
+brain coral,
+flatworm, platyhelminth,
+nematode, nematode worm, roundworm,
+conch,
+snail,
+slug,
+sea slug, nudibranch,
+chiton, coat-of-mail shell, sea cradle, polyplacophore,
+chambered nautilus, pearly nautilus, nautilus,
+Dungeness crab, Cancer magister,
+rock crab, Cancer irroratus,
+fiddler crab,
+king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica,
+American lobster, Northern lobster, Maine lobster, Homarus americanus,
+spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish,
+crayfish, crawfish, crawdad, crawdaddy,
+hermit crab,
+isopod,
+white stork, Ciconia ciconia,
+black stork, Ciconia nigra,
+spoonbill,
+flamingo,
+little blue heron, Egretta caerulea,
+American egret, great white heron, Egretta albus,
+bittern,
+crane,
+limpkin, Aramus pictus,
+European gallinule, Porphyrio porphyrio,
+American coot, marsh hen, mud hen, water hen, Fulica americana,
+bustard,
+ruddy turnstone, Arenaria interpres,
+red-backed sandpiper, dunlin, Erolia alpina,
+redshank, Tringa totanus,
+dowitcher,
+oystercatcher, oyster catcher,
+pelican,
+king penguin, Aptenodytes patagonica,
+albatross, mollymawk,
+grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus,
+killer whale, killer, orca, grampus, sea wolf, Orcinus orca,
+dugong, Dugong dugon,
+sea lion,
+Chihuahua,
+Japanese spaniel,
+Maltese dog, Maltese terrier, Maltese,
+Pekinese, Pekingese, Peke,
+Shih-Tzu,
+Blenheim spaniel,
+papillon,
+toy terrier,
+Rhodesian ridgeback,
+Afghan hound, Afghan,
+basset, basset hound,
+beagle,
+bloodhound, sleuthhound,
+bluetick,
+black-and-tan coonhound,
+Walker hound, Walker foxhound,
+English foxhound,
+redbone,
+borzoi, Russian wolfhound,
+Irish wolfhound,
+Italian greyhound,
+whippet,
+Ibizan hound, Ibizan Podenco,
+Norwegian elkhound, elkhound,
+otterhound, otter hound,
+Saluki, gazelle hound,
+Scottish deerhound, deerhound,
+Weimaraner,
+Staffordshire bullterrier, Staffordshire bull terrier,
+American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier,
+Bedlington terrier,
+Border terrier,
+Kerry blue terrier,
+Irish terrier,
+Norfolk terrier,
+Norwich terrier,
+Yorkshire terrier,
+wire-haired fox terrier,
+Lakeland terrier,
+Sealyham terrier, Sealyham,
+Airedale, Airedale terrier,
+cairn, cairn terrier,
+Australian terrier,
+Dandie Dinmont, Dandie Dinmont terrier,
+Boston bull, Boston terrier,
+miniature schnauzer,
+giant schnauzer,
+standard schnauzer,
+Scotch terrier, Scottish terrier, Scottie,
+Tibetan terrier, chrysanthemum dog,
+silky terrier, Sydney silky,
+soft-coated wheaten terrier,
+West Highland white terrier,
+Lhasa, Lhasa apso,
+flat-coated retriever,
+curly-coated retriever,
+golden retriever,
+Labrador retriever,
+Chesapeake Bay retriever,
+German short-haired pointer,
+vizsla, Hungarian pointer,
+English setter,
+Irish setter, red setter,
+Gordon setter,
+Brittany spaniel,
+clumber, clumber spaniel,
+English springer, English springer spaniel,
+Welsh springer spaniel,
+cocker spaniel, English cocker spaniel, cocker,
+Sussex spaniel,
+Irish water spaniel,
+kuvasz,
+schipperke,
+groenendael,
+malinois,
+briard,
+kelpie,
+komondor,
+Old English sheepdog, bobtail,
+Shetland sheepdog, Shetland sheep dog, Shetland,
+collie,
+Border collie,
+Bouvier des Flandres, Bouviers des Flandres,
+Rottweiler,
+German shepherd, German shepherd dog, German police dog, alsatian,
+Doberman, Doberman pinscher,
+miniature pinscher,
+Greater Swiss Mountain dog,
+Bernese mountain dog,
+Appenzeller,
+EntleBucher,
+boxer,
+bull mastiff,
+Tibetan mastiff,
+French bulldog,
+Great Dane,
+Saint Bernard, St Bernard,
+Eskimo dog, husky,
+malamute, malemute, Alaskan malamute,
+Siberian husky,
+dalmatian, coach dog, carriage dog,
+affenpinscher, monkey pinscher, monkey dog,
+basenji,
+pug, pug-dog,
+Leonberg,
+Newfoundland, Newfoundland dog,
+Great Pyrenees,
+Samoyed, Samoyede,
+Pomeranian,
+chow, chow chow,
+keeshond,
+Brabancon griffon,
+Pembroke, Pembroke Welsh corgi,
+Cardigan, Cardigan Welsh corgi,
+toy poodle,
+miniature poodle,
+standard poodle,
+Mexican hairless,
+timber wolf, grey wolf, gray wolf, Canis lupus,
+white wolf, Arctic wolf, Canis lupus tundrarum,
+red wolf, maned wolf, Canis rufus, Canis niger,
+coyote, prairie wolf, brush wolf, Canis latrans,
+dingo, warrigal, warragal, Canis dingo,
+dhole, Cuon alpinus,
+African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus,
+hyena, hyaena,
+red fox, Vulpes vulpes,
+kit fox, Vulpes macrotis,
+Arctic fox, white fox, Alopex lagopus,
+grey fox, gray fox, Urocyon cinereoargenteus,
+tabby, tabby cat,
+tiger cat,
+Persian cat,
+Siamese cat, Siamese,
+Egyptian cat,
+cougar, puma, catamount, mountain lion, painter, panther, Felis concolor,
+lynx, catamount,
+leopard, Panthera pardus,
+snow leopard, ounce, Panthera uncia,
+jaguar, panther, Panthera onca, Felis onca,
+lion, king of beasts, Panthera leo,
+tiger, Panthera tigris,
+cheetah, chetah, Acinonyx jubatus,
+brown bear, bruin, Ursus arctos,
+American black bear, black bear, Ursus americanus, Euarctos americanus,
+ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus,
+sloth bear, Melursus ursinus, Ursus ursinus,
+mongoose,
+meerkat, mierkat,
+tiger beetle,
+ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle,
+ground beetle, carabid beetle,
+long-horned beetle, longicorn, longicorn beetle,
+leaf beetle, chrysomelid,
+dung beetle,
+rhinoceros beetle,
+weevil,
+fly,
+bee,
+ant, emmet, pismire,
+grasshopper, hopper,
+cricket,
+walking stick, walkingstick, stick insect,
+cockroach, roach,
+mantis, mantid,
+cicada, cicala,
+leafhopper,
+lacewing, lacewing fly,
+"dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+damselfly,
+admiral,
+ringlet, ringlet butterfly,
+monarch, monarch butterfly, milkweed butterfly, Danaus plexippus,
+cabbage butterfly,
+sulphur butterfly, sulfur butterfly,
+lycaenid, lycaenid butterfly,
+starfish, sea star,
+sea urchin,
+sea cucumber, holothurian,
+wood rabbit, cottontail, cottontail rabbit,
+hare,
+Angora, Angora rabbit,
+hamster,
+porcupine, hedgehog,
+fox squirrel, eastern fox squirrel, Sciurus niger,
+marmot,
+beaver,
+guinea pig, Cavia cobaya,
+sorrel,
+zebra,
+hog, pig, grunter, squealer, Sus scrofa,
+wild boar, boar, Sus scrofa,
+warthog,
+hippopotamus, hippo, river horse, Hippopotamus amphibius,
+ox,
+water buffalo, water ox, Asiatic buffalo, Bubalus bubalis,
+bison,
+ram, tup,
+bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis,
+ibex, Capra ibex,
+hartebeest,
+impala, Aepyceros melampus,
+gazelle,
+Arabian camel, dromedary, Camelus dromedarius,
+llama,
+weasel,
+mink,
+polecat, fitch, foulmart, foumart, Mustela putorius,
+black-footed ferret, ferret, Mustela nigripes,
+otter,
+skunk, polecat, wood pussy,
+badger,
+armadillo,
+three-toed sloth, ai, Bradypus tridactylus,
+orangutan, orang, orangutang, Pongo pygmaeus,
+gorilla, Gorilla gorilla,
+chimpanzee, chimp, Pan troglodytes,
+gibbon, Hylobates lar,
+siamang, Hylobates syndactylus, Symphalangus syndactylus,
+guenon, guenon monkey,
+patas, hussar monkey, Erythrocebus patas,
+baboon,
+macaque,
+langur,
+colobus, colobus monkey,
+proboscis monkey, Nasalis larvatus,
+marmoset,
+capuchin, ringtail, Cebus capucinus,
+howler monkey, howler,
+titi, titi monkey,
+spider monkey, Ateles geoffroyi,
+squirrel monkey, Saimiri sciureus,
+Madagascar cat, ring-tailed lemur, Lemur catta,
+indri, indris, Indri indri, Indri brevicaudatus,
+Indian elephant, Elephas maximus,
+African elephant, Loxodonta africana,
+lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens,
+giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca,
+barracouta, snoek,
+eel,
+coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch,
+rock beauty, Holocanthus tricolor,
+anemone fish,
+sturgeon,
+gar, garfish, garpike, billfish, Lepisosteus osseus,
+lionfish,
+puffer, pufferfish, blowfish, globefish,
+abacus,
+abaya,
+"academic gown, academic robe, judges robe",
+accordion, piano accordion, squeeze box,
+acoustic guitar,
+aircraft carrier, carrier, flattop, attack aircraft carrier,
+airliner,
+airship, dirigible,
+altar,
+ambulance,
+amphibian, amphibious vehicle,
+analog clock,
+apiary, bee house,
+apron,
+ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin,
+assault rifle, assault gun,
+backpack, back pack, knapsack, packsack, rucksack, haversack,
+bakery, bakeshop, bakehouse,
+balance beam, beam,
+balloon,
+ballpoint, ballpoint pen, ballpen, Biro,
+Band Aid,
+banjo,
+bannister, banister, balustrade, balusters, handrail,
+barbell,
+barber chair,
+barbershop,
+barn,
+barometer,
+barrel, cask,
+barrow, garden cart, lawn cart, wheelbarrow,
+baseball,
+basketball,
+bassinet,
+bassoon,
+bathing cap, swimming cap,
+bath towel,
+bathtub, bathing tub, bath, tub,
+beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon,
+beacon, lighthouse, beacon light, pharos,
+beaker,
+bearskin, busby, shako,
+beer bottle,
+beer glass,
+bell cote, bell cot,
+bib,
+bicycle-built-for-two, tandem bicycle, tandem,
+bikini, two-piece,
+binder, ring-binder,
+binoculars, field glasses, opera glasses,
+birdhouse,
+boathouse,
+bobsled, bobsleigh, bob,
+bolo tie, bolo, bola tie, bola,
+bonnet, poke bonnet,
+bookcase,
+bookshop, bookstore, bookstall,
+bottlecap,
+bow,
+bow tie, bow-tie, bowtie,
+brass, memorial tablet, plaque,
+brassiere, bra, bandeau,
+breakwater, groin, groyne, mole, bulwark, seawall, jetty,
+breastplate, aegis, egis,
+broom,
+bucket, pail,
+buckle,
+bulletproof vest,
+bullet train, bullet,
+butcher shop, meat market,
+cab, hack, taxi, taxicab,
+caldron, cauldron,
+candle, taper, wax light,
+cannon,
+canoe,
+can opener, tin opener,
+cardigan,
+car mirror,
+carousel, carrousel, merry-go-round, roundabout, whirligig,
+"carpenters kit, tool kit",
+carton,
+car wheel,
+cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM,
+cassette,
+cassette player,
+castle,
+catamaran,
+CD player,
+cello, violoncello,
+cellular telephone, cellular phone, cellphone, cell, mobile phone,
+chain,
+chainlink fence,
+chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour,
+chain saw, chainsaw,
+chest,
+chiffonier, commode,
+chime, bell, gong,
+china cabinet, china closet,
+Christmas stocking,
+church, church building,
+cinema, movie theater, movie theatre, movie house, picture palace,
+cleaver, meat cleaver, chopper,
+cliff dwelling,
+cloak,
+clog, geta, patten, sabot,
+cocktail shaker,
+coffee mug,
+coffeepot,
+coil, spiral, volute, whorl, helix,
+combination lock,
+computer keyboard, keypad,
+confectionery, confectionary, candy store,
+container ship, containership, container vessel,
+convertible,
+corkscrew, bottle screw,
+cornet, horn, trumpet, trump,
+cowboy boot,
+cowboy hat, ten-gallon hat,
+cradle,
+crane,
+crash helmet,
+crate,
+crib, cot,
+Crock Pot,
+croquet ball,
+crutch,
+cuirass,
+dam, dike, dyke,
+desk,
+desktop computer,
+dial telephone, dial phone,
+diaper, nappy, napkin,
+digital clock,
+digital watch,
+dining table, board,
+dishrag, dishcloth,
+dishwasher, dish washer, dishwashing machine,
+disk brake, disc brake,
+dock, dockage, docking facility,
+dogsled, dog sled, dog sleigh,
+dome,
+doormat, welcome mat,
+drilling platform, offshore rig,
+drum, membranophone, tympan,
+drumstick,
+dumbbell,
+Dutch oven,
+electric fan, blower,
+electric guitar,
+electric locomotive,
+entertainment center,
+envelope,
+espresso maker,
+face powder,
+feather boa, boa,
+file, file cabinet, filing cabinet,
+fireboat,
+fire engine, fire truck,
+fire screen, fireguard,
+flagpole, flagstaff,
+flute, transverse flute,
+folding chair,
+football helmet,
+forklift,
+fountain,
+fountain pen,
+four-poster,
+freight car,
+French horn, horn,
+frying pan, frypan, skillet,
+fur coat,
+garbage truck, dustcart,
+gasmask, respirator, gas helmet,
+gas pump, gasoline pump, petrol pump, island dispenser,
+goblet,
+go-kart,
+golf ball,
+golfcart, golf cart,
+gondola,
+gong, tam-tam,
+gown,
+grand piano, grand,
+greenhouse, nursery, glasshouse,
+grille, radiator grille,
+grocery store, grocery, food market, market,
+guillotine,
+hair slide,
+hair spray,
+half track,
+hammer,
+hamper,
+hand blower, blow dryer, blow drier, hair dryer, hair drier,
+hand-held computer, hand-held microcomputer,
+handkerchief, hankie, hanky, hankey,
+hard disc, hard disk, fixed disk,
+harmonica, mouth organ, harp, mouth harp,
+harp,
+harvester, reaper,
+hatchet,
+holster,
+home theater, home theatre,
+honeycomb,
+hook, claw,
+hoopskirt, crinoline,
+horizontal bar, high bar,
+horse cart, horse-cart,
+hourglass,
+iPod,
+iron, smoothing iron,
+"jack-o-lantern",
+jean, blue jean, denim,
+jeep, landrover,
+jersey, T-shirt, tee shirt,
+jigsaw puzzle,
+jinrikisha, ricksha, rickshaw,
+joystick,
+kimono,
+knee pad,
+knot,
+lab coat, laboratory coat,
+ladle,
+lampshade, lamp shade,
+laptop, laptop computer,
+lawn mower, mower,
+lens cap, lens cover,
+letter opener, paper knife, paperknife,
+library,
+lifeboat,
+lighter, light, igniter, ignitor,
+limousine, limo,
+liner, ocean liner,
+lipstick, lip rouge,
+Loafer,
+lotion,
+loudspeaker, speaker, speaker unit, loudspeaker system, speaker system,
+"loupe, jewelers loupe",
+lumbermill, sawmill,
+magnetic compass,
+mailbag, postbag,
+mailbox, letter box,
+maillot,
+maillot, tank suit,
+manhole cover,
+maraca,
+marimba, xylophone,
+mask,
+matchstick,
+maypole,
+maze, labyrinth,
+measuring cup,
+medicine chest, medicine cabinet,
+megalith, megalithic structure,
+microphone, mike,
+microwave, microwave oven,
+military uniform,
+milk can,
+minibus,
+miniskirt, mini,
+minivan,
+missile,
+mitten,
+mixing bowl,
+mobile home, manufactured home,
+Model T,
+modem,
+monastery,
+monitor,
+moped,
+mortar,
+mortarboard,
+mosque,
+mosquito net,
+motor scooter, scooter,
+mountain bike, all-terrain bike, off-roader,
+mountain tent,
+mouse, computer mouse,
+mousetrap,
+moving van,
+muzzle,
+nail,
+neck brace,
+necklace,
+nipple,
+notebook, notebook computer,
+obelisk,
+oboe, hautboy, hautbois,
+ocarina, sweet potato,
+odometer, hodometer, mileometer, milometer,
+oil filter,
+organ, pipe organ,
+oscilloscope, scope, cathode-ray oscilloscope, CRO,
+overskirt,
+oxcart,
+oxygen mask,
+packet,
+paddle, boat paddle,
+paddlewheel, paddle wheel,
+padlock,
+paintbrush,
+"pajama, pyjama, pjs, jammies",
+palace,
+panpipe, pandean pipe, syrinx,
+paper towel,
+parachute, chute,
+parallel bars, bars,
+park bench,
+parking meter,
+passenger car, coach, carriage,
+patio, terrace,
+pay-phone, pay-station,
+pedestal, plinth, footstall,
+pencil box, pencil case,
+pencil sharpener,
+perfume, essence,
+Petri dish,
+photocopier,
+pick, plectrum, plectron,
+pickelhaube,
+picket fence, paling,
+pickup, pickup truck,
+pier,
+piggy bank, penny bank,
+pill bottle,
+pillow,
+ping-pong ball,
+pinwheel,
+pirate, pirate ship,
+pitcher, ewer,
+"plane, carpenters plane, woodworking plane",
+planetarium,
+plastic bag,
+plate rack,
+plow, plough,
+"plunger, plumbers helper",
+Polaroid camera, Polaroid Land camera,
+pole,
+police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria,
+poncho,
+pool table, billiard table, snooker table,
+pop bottle, soda bottle,
+pot, flowerpot,
+"potters wheel",
+power drill,
+prayer rug, prayer mat,
+printer,
+prison, prison house,
+projectile, missile,
+projector,
+puck, hockey puck,
+punching bag, punch bag, punching ball, punchball,
+purse,
+quill, quill pen,
+quilt, comforter, comfort, puff,
+racer, race car, racing car,
+racket, racquet,
+radiator,
+radio, wireless,
+radio telescope, radio reflector,
+rain barrel,
+recreational vehicle, RV, R.V.,
+reel,
+reflex camera,
+refrigerator, icebox,
+remote control, remote,
+restaurant, eating house, eating place, eatery,
+revolver, six-gun, six-shooter,
+rifle,
+rocking chair, rocker,
+rotisserie,
+rubber eraser, rubber, pencil eraser,
+rugby ball,
+rule, ruler,
+running shoe,
+safe,
+safety pin,
+saltshaker, salt shaker,
+sandal,
+sarong,
+sax, saxophone,
+scabbard,
+scale, weighing machine,
+school bus,
+schooner,
+scoreboard,
+screen, CRT screen,
+screw,
+screwdriver,
+seat belt, seatbelt,
+sewing machine,
+shield, buckler,
+shoe shop, shoe-shop, shoe store,
+shoji,
+shopping basket,
+shopping cart,
+shovel,
+shower cap,
+shower curtain,
+ski,
+ski mask,
+sleeping bag,
+slide rule, slipstick,
+sliding door,
+slot, one-armed bandit,
+snorkel,
+snowmobile,
+snowplow, snowplough,
+soap dispenser,
+soccer ball,
+sock,
+solar dish, solar collector, solar furnace,
+sombrero,
+soup bowl,
+space bar,
+space heater,
+space shuttle,
+spatula,
+speedboat,
+"spider web, spiders web",
+spindle,
+sports car, sport car,
+spotlight, spot,
+stage,
+steam locomotive,
+steel arch bridge,
+steel drum,
+stethoscope,
+stole,
+stone wall,
+stopwatch, stop watch,
+stove,
+strainer,
+streetcar, tram, tramcar, trolley, trolley car,
+stretcher,
+studio couch, day bed,
+stupa, tope,
+submarine, pigboat, sub, U-boat,
+suit, suit of clothes,
+sundial,
+sunglass,
+sunglasses, dark glasses, shades,
+sunscreen, sunblock, sun blocker,
+suspension bridge,
+swab, swob, mop,
+sweatshirt,
+swimming trunks, bathing trunks,
+swing,
+switch, electric switch, electrical switch,
+syringe,
+table lamp,
+tank, army tank, armored combat vehicle, armoured combat vehicle,
+tape player,
+teapot,
+teddy, teddy bear,
+television, television system,
+tennis ball,
+thatch, thatched roof,
+theater curtain, theatre curtain,
+thimble,
+thresher, thrasher, threshing machine,
+throne,
+tile roof,
+toaster,
+tobacco shop, tobacconist shop, tobacconist,
+toilet seat,
+torch,
+totem pole,
+tow truck, tow car, wrecker,
+toyshop,
+tractor,
+trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi,
+tray,
+trench coat,
+tricycle, trike, velocipede,
+trimaran,
+tripod,
+triumphal arch,
+trolleybus, trolley coach, trackless trolley,
+trombone,
+tub, vat,
+turnstile,
+typewriter keyboard,
+umbrella,
+unicycle, monocycle,
+upright, upright piano,
+vacuum, vacuum cleaner,
+vase,
+vault,
+velvet,
+vending machine,
+vestment,
+viaduct,
+violin, fiddle,
+volleyball,
+waffle iron,
+wall clock,
+wallet, billfold, notecase, pocketbook,
+wardrobe, closet, press,
+warplane, military plane,
+washbasin, handbasin, washbowl, lavabo, wash-hand basin,
+washer, automatic washer, washing machine,
+water bottle,
+water jug,
+water tower,
+whiskey jug,
+whistle,
+wig,
+window screen,
+window shade,
+Windsor tie,
+wine bottle,
+wing,
+wok,
+wooden spoon,
+wool, woolen, woollen,
+worm fence, snake fence, snake-rail fence, Virginia fence,
+wreck,
+yawl,
+yurt,
+web site, website, internet site, site,
+comic book,
+crossword puzzle, crossword,
+street sign,
+traffic light, traffic signal, stoplight,
+book jacket, dust cover, dust jacket, dust wrapper,
+menu,
+plate,
+guacamole,
+consomme,
+hot pot, hotpot,
+trifle,
+ice cream, icecream,
+ice lolly, lolly, lollipop, popsicle,
+French loaf,
+bagel, beigel,
+pretzel,
+cheeseburger,
+hotdog, hot dog, red hot,
+mashed potato,
+head cabbage,
+broccoli,
+cauliflower,
+zucchini, courgette,
+spaghetti squash,
+acorn squash,
+butternut squash,
+cucumber, cuke,
+artichoke, globe artichoke,
+bell pepper,
+cardoon,
+mushroom,
+Granny Smith,
+strawberry,
+orange,
+lemon,
+fig,
+pineapple, ananas,
+banana,
+jackfruit, jak, jack,
+custard apple,
+pomegranate,
+hay,
+carbonara,
+chocolate sauce, chocolate syrup,
+dough,
+meat loaf, meatloaf,
+pizza, pizza pie,
+potpie,
+burrito,
+red wine,
+espresso,
+cup,
+eggnog,
+alp,
+bubble,
+cliff, drop, drop-off,
+coral reef,
+geyser,
+lakeside, lakeshore,
+promontory, headland, head, foreland,
+sandbar, sand bar,
+seashore, coast, seacoast, sea-coast,
+valley, vale,
+volcano,
+ballplayer, baseball player,
+groom, bridegroom,
+scuba diver,
+rapeseed,
+daisy,
+"yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+corn,
+acorn,
+hip, rose hip, rosehip,
+buckeye, horse chestnut, conker,
+coral fungus,
+agaric,
+gyromitra,
+stinkhorn, carrion fungus,
+earthstar,
+hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa,
+bolete,
+ear, spike, capitulum,
+toilet tissue, toilet paper, bathroom tissue
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_http_client.py b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_http_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..bc3fab2524cbb23f2f398d3effb667d3629363c7
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_http_client.py
@@ -0,0 +1,19 @@
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+if __name__ == "__main__":
+ url = "http://127.0.0.1:18080/imagenet/prediction"
+ with open(os.path.join(".", "daisy.jpg"), 'rb') as file:
+ image_data1 = file.read()
+ image = cv2_to_base64(image_data1)
+ data = {"key": ["image"], "value": [image]}
+ for i in range(100):
+ r = requests.post(url=url, data=json.dumps(data))
+ print(r.json())
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_rpc_client.py b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_rpc_client.py
new file mode 100644
index 0000000000000000000000000000000000000000..34a08f4b5d1ec2861c3101685b434453d61156de
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/pipeline_rpc_client.py
@@ -0,0 +1,39 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+try:
+ from paddle_serving_server_gpu.pipeline import PipelineClient
+except ImportError:
+ from paddle_serving_server.pipeline import PipelineClient
+import numpy as np
+import requests
+import json
+import cv2
+import base64
+import os
+
+client = PipelineClient()
+client.connect(['127.0.0.1:9993'])
+
+
+def cv2_to_base64(image):
+ return base64.b64encode(image).decode('utf8')
+
+
+with open("daisy.jpg", 'rb') as file:
+ image_data = file.read()
+image = cv2_to_base64(image_data)
+
+for i in range(1):
+ ret = client.predict(feed_dict={"image": image}, fetch=["label", "prob"])
+ print(ret)
diff --git a/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/resnet50_web_service.py b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/resnet50_web_service.py
new file mode 100644
index 0000000000000000000000000000000000000000..6469657ac297c09f57f08c9cbafb806f62214fea
--- /dev/null
+++ b/python/examples/pipeline/PaddleClas/ShuffleNetV2_x1_0/resnet50_web_service.py
@@ -0,0 +1,74 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import sys
+from paddle_serving_app.reader import Sequential, URL2Image, Resize, CenterCrop, RGB2BGR, Transpose, Div, Normalize, Base64ToImage
+try:
+ from paddle_serving_server_gpu.web_service import WebService, Op
+except ImportError:
+ from paddle_serving_server.web_service import WebService, Op
+import logging
+import numpy as np
+import base64, cv2
+
+
+class ImagenetOp(Op):
+ def init_op(self):
+ self.seq = Sequential([
+ Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)),
+ Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225],
+ True)
+ ])
+ self.label_dict = {}
+ label_idx = 0
+ with open("imagenet.label") as fin:
+ for line in fin:
+ self.label_dict[label_idx] = line.strip()
+ label_idx += 1
+
+ def preprocess(self, input_dicts, data_id, log_id):
+ (_, input_dict), = input_dicts.items()
+ batch_size = len(input_dict.keys())
+ imgs = []
+ for key in input_dict.keys():
+ data = base64.b64decode(input_dict[key].encode('utf8'))
+ data = np.fromstring(data, np.uint8)
+ im = cv2.imdecode(data, cv2.IMREAD_COLOR)
+ img = self.seq(im)
+ imgs.append(img[np.newaxis, :].copy())
+ input_imgs = np.concatenate(imgs, axis=0)
+ return {"image": input_imgs}, False, None, ""
+
+ def postprocess(self, input_dicts, fetch_dict, log_id):
+ score_list = fetch_dict["prediction"]
+ result = {"label": [], "prob": []}
+ for score in score_list:
+ score = score.tolist()
+ max_score = max(score)
+ result["label"].append(self.label_dict[score.index(max_score)]
+ .strip().replace(",", ""))
+ result["prob"].append(max_score)
+ result["label"] = str(result["label"])
+ result["prob"] = str(result["prob"])
+ return result, None, ""
+
+
+class ImageService(WebService):
+ def get_pipeline_response(self, read_op):
+ image_op = ImagenetOp(name="imagenet", input_ops=[read_op])
+ return image_op
+
+
+uci_service = ImageService(name="imagenet")
+uci_service.prepare_pipeline_config("config.yml")
+uci_service.run_service()
diff --git a/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.py b/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.py
index e2bef4f91e98ecd8459fc0ff431a9b784d3c639a..f0a55614c1390b1d4f73bd015b1ce21b85e4ba55 100644
--- a/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.py
+++ b/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.py
@@ -46,23 +46,43 @@ def run_http(idx, batch_size):
with open(os.path.join(".", "000000570688.jpg"), 'rb') as file:
image_data1 = file.read()
image = cv2_to_base64(image_data1)
-
+ latency_list = []
start = time.time()
+ total_num = 0
while True:
+ l_start = time.time()
data = {"key": [], "value": []}
for j in range(batch_size):
data["key"].append("image_" + str(j))
data["value"].append(image)
r = requests.post(url=url, data=json.dumps(data))
+ l_end = time.time()
+ total_num += 1
end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
if end - start > 70:
- print("70s end")
+ #print("70s end")
break
- return [[end - start]]
+ return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner()
- result = multi_thread_runner.run(run_http , thread, batch_size)
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
def run_rpc(thread, batch_size):
pass
diff --git a/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.sh b/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.sh
index f67caaa7761929a48579b5d5ed56fa93bb3290ae..5247891f64f7a669a20e2ad19fa1f4cb94e1fb17 100644
--- a/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.sh
+++ b/python/examples/pipeline/PaddleDetection/faster_rcnn/benchmark.sh
@@ -1,36 +1,44 @@
export FLAGS_profile_pipeline=1
-alias python3="python3.7"
-modelname="faster_rcnn_r50_fpn_1x_coco"
-gpu_id="0"
-benchmark_config_filename="benchmark_config.yaml"
+alias python3="python3.6"
+modelname="det-FasterRCNN"
+
# HTTP
-ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
sleep 3
-python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
rm -rf profile_log_$modelname
-for thread_num in 1
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
do
- for batch_size in 1
+ for batch_size in 1
do
- echo "#----FasterRCNN thread num: $thread_num batch size: $batch_size mode:http ----" >>profile_log_$modelname
- rm -rf PipelineServingLogs
- rm -rf cpu_utilization.py
- python3 web_service.py >web.log 2>&1 &
- sleep 3
- nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
- nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
- python3 benchmark.py run http $thread_num $batch_size
- python3 cpu_utilization.py >>profile_log_$modelname
- python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
- ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
- ps -ef | grep nvidia-smi | awk '{print $2}' | xargs kill -9
- python3 benchmark.py dump benchmark.log benchmark.tmp
- mv benchmark.tmp benchmark.log
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
- cat benchmark.log >> profile_log_$modelname
- python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
- #rm -rf gpu_use.log gpu_utilization.log
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
done
done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleDetection/faster_rcnn/web_service.py b/python/examples/pipeline/PaddleDetection/faster_rcnn/web_service.py
index 6334ef7242134af63ff7e6911d7f992c30289207..691d647befa8e1c583a53121e89ab5f2859f64b7 100644
--- a/python/examples/pipeline/PaddleDetection/faster_rcnn/web_service.py
+++ b/python/examples/pipeline/PaddleDetection/faster_rcnn/web_service.py
@@ -64,5 +64,5 @@ class FasterRCNNService(WebService):
fasterrcnn_service = FasterRCNNService(name="faster_rcnn")
-fasterrcnn_service.prepare_pipeline_config("config2.yml")
+fasterrcnn_service.prepare_pipeline_config("config.yml")
fasterrcnn_service.run_service()
diff --git a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.py b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.py
index 714acb12ef8451b967d44b05524791e07ad92056..a23f64314ef448f2617f92ab40f94f75cc6e707f 100644
--- a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.py
+++ b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.py
@@ -36,7 +36,7 @@ def gen_yml(device, gpu_id):
config["dag"]["tracer"] = {"interval_s": 30}
if device == "gpu":
config["op"]["ppyolo_mbv3"]["local_service_conf"]["device_type"] = 1
- config["op"]["ppyolo_mbv3"]["local_service_conf"]["devices"] = gpu_id
+ config["op"]["ppyolo_mbv3"]["local_service_conf"]["devices"] = gpu_id
with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False)
@@ -46,23 +46,43 @@ def run_http(idx, batch_size):
with open(os.path.join(".", "000000570688.jpg"), 'rb') as file:
image_data1 = file.read()
image = cv2_to_base64(image_data1)
-
+ latency_list = []
start = time.time()
+ total_num = 0
while True:
+ l_start = time.time()
data = {"key": [], "value": []}
for j in range(batch_size):
data["key"].append("image_" + str(j))
data["value"].append(image)
r = requests.post(url=url, data=json.dumps(data))
+ l_end = time.time()
+ total_num += 1
end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
if end - start > 70:
- print("70s end")
+ #print("70s end")
break
- return [[end - start]]
+ return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner()
- result = multi_thread_runner.run(run_http , thread, batch_size)
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
def run_rpc(thread, batch_size):
pass
diff --git a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.sh b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.sh
index 1a7cb64429b7f20f1bd01f191518f53b1f82e40c..7237a52ab73630141785f47a143d6931f75d0c17 100644
--- a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.sh
+++ b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/benchmark.sh
@@ -1,36 +1,44 @@
export FLAGS_profile_pipeline=1
-alias python3="python3.7"
-modelname="ppyolo_mbv3_large"
-gpu_id="0"
-benchmark_config_filename="benchmark_config.yaml"
+alias python3="python3.6"
+modelname="det-PPYoloMbv3"
# HTTP
-ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
sleep 3
-python3 benchmark.py yaml local_predictor 1 gpu $gpu_id
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
rm -rf profile_log_$modelname
-for thread_num in 1
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
do
- for batch_size in 1
+ for batch_size in 1
do
- echo "#----PPyolo thread num: $thread_num batch size: $batch_size mode:http ----" >>profile_log_$modelname
- rm -rf PipelineServingLogs
- rm -rf cpu_utilization.py
- python3 web_service.py >web.log 2>&1 &
- sleep 3
- nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
- nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
- python3 benchmark.py run http $thread_num $batch_size
- python3 cpu_utilization.py >>profile_log_$modelname
- python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
- ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
- python3 benchmark.py dump benchmark.log benchmark.tmp
- mv benchmark.tmp benchmark.log
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
- cat benchmark.log >> profile_log_$modelname
- python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
- #rm -rf gpu_use.log gpu_utilization.log
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
done
done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/web_service.py b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/web_service.py
index aa96b2faf9ca07956e6da0b61e0228cd27cf1d60..8611b0671862a887efd1705b3c1a922db906581d 100644
--- a/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/web_service.py
+++ b/python/examples/pipeline/PaddleDetection/ppyolo_mbv3/web_service.py
@@ -65,5 +65,5 @@ class PPYoloMbv(WebService):
ppyolo_mbv3_service = PPYoloMbv(name="ppyolo_mbv3")
-ppyolo_mbv3_service.prepare_pipeline_config("config2.yml")
+ppyolo_mbv3_service.prepare_pipeline_config("config.yml")
ppyolo_mbv3_service.run_service()
diff --git a/python/examples/pipeline/PaddleDetection/yolov3/benchmark.py b/python/examples/pipeline/PaddleDetection/yolov3/benchmark.py
index 2918c111e4a90b1c460780f6ecb7cecc6df09335..ae9c5a8fb25f56eebe3c3893a4a4d251f21e5b61 100644
--- a/python/examples/pipeline/PaddleDetection/yolov3/benchmark.py
+++ b/python/examples/pipeline/PaddleDetection/yolov3/benchmark.py
@@ -35,8 +35,8 @@ def gen_yml(device, gpu_id):
fin.close()
config["dag"]["tracer"] = {"interval_s": 30}
if device == "gpu":
- config["op"]["faster_rcnn"]["local_service_conf"]["device_type"] = 1
- config["op"]["faster_rcnn"]["local_service_conf"]["devices"] = gpu_id
+ config["op"]["yolov3"]["local_service_conf"]["device_type"] = 1
+ config["op"]["yolov3"]["local_service_conf"]["devices"] = gpu_id
with open("config2.yml", "w") as fout:
yaml.dump(config, fout, default_flow_style=False)
@@ -46,23 +46,43 @@ def run_http(idx, batch_size):
with open(os.path.join(".", "000000570688.jpg"), 'rb') as file:
image_data1 = file.read()
image = cv2_to_base64(image_data1)
-
+ latency_list = []
start = time.time()
+ total_num = 0
while True:
+ l_start = time.time()
data = {"key": [], "value": []}
for j in range(batch_size):
data["key"].append("image_" + str(j))
data["value"].append(image)
r = requests.post(url=url, data=json.dumps(data))
+ l_end = time.time()
+ total_num += 1
end = time.time()
+ latency_list.append(l_end * 1000 - l_start * 1000)
if end - start > 70:
- print("70s end")
+ #print("70s end")
break
- return [[end - start]]
+ return [[end - start], latency_list, [total_num]]
def multithread_http(thread, batch_size):
multi_thread_runner = MultiThreadRunner()
- result = multi_thread_runner.run(run_http , thread, batch_size)
+ start = time.time()
+ result = multi_thread_runner.run(run_http, thread, batch_size)
+ end = time.time()
+ total_cost = end - start
+ avg_cost = 0
+ total_number = 0
+ for i in range(thread):
+ avg_cost += result[0][i]
+ total_number += result[2][i]
+ avg_cost = avg_cost / thread
+ print("Total cost: {}s".format(total_cost))
+ print("Each thread cost: {}s. ".format(avg_cost))
+ print("Total count: {}. ".format(total_number))
+ print("AVG QPS: {} samples/s".format(batch_size * total_number /
+ total_cost))
+ show_latency(result[1])
def run_rpc(thread, batch_size):
pass
diff --git a/python/examples/pipeline/PaddleDetection/yolov3/benchmark.sh b/python/examples/pipeline/PaddleDetection/yolov3/benchmark.sh
index 74ab42bd55cd6b9d26c593d6e13d97ff99b3c4e9..e3ac2f79398feea88a1d547047af6db0691c1581 100644
--- a/python/examples/pipeline/PaddleDetection/yolov3/benchmark.sh
+++ b/python/examples/pipeline/PaddleDetection/yolov3/benchmark.sh
@@ -1,36 +1,44 @@
export FLAGS_profile_pipeline=1
-alias python3="python3.7"
-modelname="yolov3_darknet53_270e_coco"
-gpu_id="0"
-benchmark_config_filename="benchmark_config.yaml"
+alias python3="python3.6"
+modelname="det-yolov3"
# HTTP
-ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
+#ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
sleep 3
-python3 benchmark.py yaml local_predictor 1 cpu
+# Create yaml,If you already have the config.yaml, ignore it.
+#python3 benchmark.py yaml local_predictor 1 gpu
rm -rf profile_log_$modelname
-for thread_num in 1 8 16
+
+echo "Starting HTTP Clients..."
+# Start a client in each thread, tesing the case of multiple threads.
+for thread_num in 1 2 4 8 12 16
do
- for batch_size in 1
+ for batch_size in 1
do
- echo "#----Yolov3 thread num: $thread_num batch size: $batch_size mode:http ----" >>profile_log_$modelname
- rm -rf PipelineServingLogs
- rm -rf cpu_utilization.py
- python3 web_service.py >web.log 2>&1 &
- sleep 3
- nvidia-smi --id=${gpu_id} --query-compute-apps=used_memory --format=csv -lms 100 > gpu_use.log 2>&1 &
- nvidia-smi --id=${gpu_id} --query-gpu=utilization.gpu --format=csv -lms 100 > gpu_utilization.log 2>&1 &
+ echo "----${modelname} thread num: ${thread_num} batch size: ${batch_size} mode:http ----" >>profile_log_$modelname
+ # Start one web service, If you start the service yourself, you can ignore it here.
+ #python3 web_service.py >web.log 2>&1 &
+ #sleep 3
+
+ # --id is the serial number of the GPU card, Must be the same as the gpu id used by the server.
+ nvidia-smi --id=3 --query-gpu=memory.used --format=csv -lms 1000 > gpu_use.log 2>&1 &
+ nvidia-smi --id=3 --query-gpu=utilization.gpu --format=csv -lms 1000 > gpu_utilization.log 2>&1 &
echo "import psutil\ncpu_utilization=psutil.cpu_percent(1,False)\nprint('CPU_UTILIZATION:', cpu_utilization)\n" > cpu_utilization.py
- python3 benchmark.py run http $thread_num $batch_size
- python3 cpu_utilization.py >>profile_log_$modelname
- python3 -m paddle_serving_server_gpu.profiler >>profile_log_$modelname
- ps -ef | grep web_service | awk '{print $2}' | xargs kill -9
- python3 benchmark.py dump benchmark.log benchmark.tmp
- mv benchmark.tmp benchmark.log
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_MEM:", max}' gpu_use.log >> profile_log_$modelname
- awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "GPU_UTIL:", max}' gpu_utilization.log >> profile_log_$modelname
- cat benchmark.log >> profile_log_$modelname
- python3 -m paddle_serving_server_gpu.parse_profile --benchmark_cfg $benchmark_config_filename --benchmark_log profile_log_$modelname
- #rm -rf gpu_use.log gpu_utilization.log
+ # Start http client
+ python3 benchmark.py run http $thread_num $batch_size > profile 2>&1
+
+ # Collect CPU metrics, Filter data that is zero momentarily, Record the maximum value of GPU memory and the average value of GPU utilization
+ python3 cpu_utilization.py >> profile_log_$modelname
+ grep -av '^0 %' gpu_utilization.log > gpu_utilization.log.tmp
+ awk 'BEGIN {max = 0} {if(NR>1){if ($modelname > max) max=$modelname}} END {print "MAX_GPU_MEMORY:", max}' gpu_use.log >> profile_log_$modelname
+ awk -F' ' '{sum+=$1} END {print "GPU_UTILIZATION:", sum/NR, sum, NR }' gpu_utilization.log.tmp >> profile_log_$modelname
+
+ # Show profiles
+ python3 ../../../util/show_profile.py profile $thread_num >> profile_log_$modelname
+ tail -n 8 profile >> profile_log_$modelname
+ echo '' >> profile_log_$modelname
done
done
+
+# Kill all nvidia-smi background task.
+pkill nvidia-smi
diff --git a/python/examples/pipeline/PaddleDetection/yolov3/web_service.py b/python/examples/pipeline/PaddleDetection/yolov3/web_service.py
index 1d477184d5296a94f70a538971d39c9033b3f23b..d28c22b9cc1060af29b6b31140911fc848bdec28 100644
--- a/python/examples/pipeline/PaddleDetection/yolov3/web_service.py
+++ b/python/examples/pipeline/PaddleDetection/yolov3/web_service.py
@@ -64,5 +64,5 @@ class Yolov3Service(WebService):
yolov3_service = Yolov3Service(name="yolov3")
-yolov3_service.prepare_pipeline_config("config2.yml")
+yolov3_service.prepare_pipeline_config("config.yml")
yolov3_service.run_service()
diff --git a/tools/auth/auth-serving-docker.yaml b/tools/auth/auth-serving-docker.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..49659563d74fc08a8d569bddb702b206b6225a51
--- /dev/null
+++ b/tools/auth/auth-serving-docker.yaml
@@ -0,0 +1,109 @@
+version: '3'
+
+volumes:
+ kong_data: {}
+
+services:
+ db:
+ image: postgres:9.6
+ environment:
+ - POSTGRES_DB=kong
+ - POSTGRES_USER=kong
+ - POSTGRES_PASSWORD=kong
+ volumes:
+ - kong_data:/var/lib/postgresql/data
+ healthcheck:
+ test: ["CMD-SHELL", "pg_isready -U postgres"]
+ interval: 10s
+ timeout: 5s
+ retries: 5
+ restart: always
+
+ kong-migrations:
+ image: registry.baidubce.com/serving_gateway/kong:paddle
+ command: kong migrations bootstrap
+ depends_on:
+ - db
+ environment:
+ - KONG_DATABASE=postgres
+ - KONG_PG_DATABASE=kong
+ - KONG_PG_HOST=db
+ - KONG_PG_USER=kong
+ - KONG_PG_PASSWORD=kong
+ restart: on-failure
+
+ kong:
+ image: registry.baidubce.com/serving_gateway/kong:paddle
+ depends_on:
+ - db
+ - redis
+ environment:
+ - KONG_DATABASE=postgres
+ - KONG_PG_HOST=db
+ - KONG_PG_DATABASE=kong
+ - KONG_PG_USER=kong
+ - KONG_PG_PASSWORD=kong
+ # - KONGKA_REDIS_HOST=redis
+ # - KONGKA_REDIS_PORT=6379
+ # - KONGKA_REDIS_DATABASE=0
+ ports:
+ - 8000:8000/tcp
+ - 127.0.0.1:8001:8001/tcp
+ - 8443:8443/tcp
+ - 127.0.0.1:8444:8444/tcp
+ healthcheck:
+ test: ["CMD", "kong", "health"]
+ interval: 10s
+ timeout: 10s
+ retries: 10
+ restart: always
+
+ kong-prepare:
+ image: registry.baidubce.com/serving_gateway/kong:paddle
+ entrypoint: ["bash", "/autoconfigure-admin-api.sh"]
+ depends_on:
+ - kong
+ restart: on-failure
+
+ konga-prepare:
+ image: pantsel/konga:next
+ command: -c prepare -a postgres -u postgresql://kong:kong@db/konga
+ depends_on:
+ - db
+ restart: on-failure
+ healthcheck:
+ test: "exit 0"
+
+ konga:
+ image: pantsel/konga:next
+ environment:
+ - DB_ADAPTER=postgres
+ - DB_HOST=db
+ - DB_USER=kong
+ - DB_PASSWORD=kong
+ - DB_DATABASE=konga
+ - NODE_ENV=production
+ depends_on:
+ - db
+ - konga-prepare
+ ports:
+ - 8005:1337/tcp
+
+ restart: always
+
+ redis:
+ image: redis:latest
+ ports:
+ - 6379:6379
+ restart: always
+
+ serving:
+ image: registry.baidubce.com/serving_dev/serving-runtime:cpu-py36
+ ports:
+ - 9393:9393
+ command: bash -c "
+ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz
+ && tar -xzf uci_housing.tar.gz
+ && python3.6 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --name uci
+ "
+ restart: always
diff --git a/tools/auth/key-auth-k8s.yaml b/tools/auth/key-auth-k8s.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..b5247c043d71c636d072913af1b917866e762d64
--- /dev/null
+++ b/tools/auth/key-auth-k8s.yaml
@@ -0,0 +1,5 @@
+apiVersion: configuration.konghq.com/v1
+kind: KongPlugin
+metadata:
+ name: key-auth
+plugin: key-auth
diff --git a/tools/auth/kong-consumer-k8s.yaml b/tools/auth/kong-consumer-k8s.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..7cfcd3276b17f24c6b0a3934dddcd3f8d4a6b616
--- /dev/null
+++ b/tools/auth/kong-consumer-k8s.yaml
@@ -0,0 +1,9 @@
+apiVersion: configuration.konghq.com/v1
+kind: KongConsumer
+metadata:
+ name: default
+ annotations:
+ kubernetes.io/ingress.class: kong
+username: default
+credentials:
+- default-apikey
diff --git a/tools/auth/kong-ingress-k8s.yaml b/tools/auth/kong-ingress-k8s.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..acc45194d7c3337c71c6655fd18b5fac2131f37e
--- /dev/null
+++ b/tools/auth/kong-ingress-k8s.yaml
@@ -0,0 +1,15 @@
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+ name: demo
+ annotations:
+ konghq.com/strip-path: "true"
+ kubernetes.io/ingress.class: kong
+spec:
+ rules:
+ - http:
+ paths:
+ - path: /foo
+ backend:
+ serviceName: uci
+ servicePort: 9393
diff --git a/tools/auth/kong-ingress.yaml b/tools/auth/kong-ingress.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..8e49f8b13e52003d408684c6c790909c7a65760a
--- /dev/null
+++ b/tools/auth/kong-ingress.yaml
@@ -0,0 +1,731 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+ name: kong
+---
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: kongclusterplugins.configuration.konghq.com
+spec:
+ additionalPrinterColumns:
+ - JSONPath: .plugin
+ description: Name of the plugin
+ name: Plugin-Type
+ type: string
+ - JSONPath: .metadata.creationTimestamp
+ description: Age
+ name: Age
+ type: date
+ - JSONPath: .disabled
+ description: Indicates if the plugin is disabled
+ name: Disabled
+ priority: 1
+ type: boolean
+ - JSONPath: .config
+ description: Configuration of the plugin
+ name: Config
+ priority: 1
+ type: string
+ group: configuration.konghq.com
+ names:
+ kind: KongClusterPlugin
+ plural: kongclusterplugins
+ shortNames:
+ - kcp
+ scope: Cluster
+ subresources:
+ status: {}
+ validation:
+ openAPIV3Schema:
+ properties:
+ config:
+ type: object
+ configFrom:
+ properties:
+ secretKeyRef:
+ properties:
+ key:
+ type: string
+ name:
+ type: string
+ namespace:
+ type: string
+ required:
+ - name
+ - namespace
+ - key
+ type: object
+ type: object
+ disabled:
+ type: boolean
+ plugin:
+ type: string
+ protocols:
+ items:
+ enum:
+ - http
+ - https
+ - grpc
+ - grpcs
+ - tcp
+ - tls
+ type: string
+ type: array
+ run_on:
+ enum:
+ - first
+ - second
+ - all
+ type: string
+ required:
+ - plugin
+ version: v1
+---
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: kongconsumers.configuration.konghq.com
+spec:
+ additionalPrinterColumns:
+ - JSONPath: .username
+ description: Username of a Kong Consumer
+ name: Username
+ type: string
+ - JSONPath: .metadata.creationTimestamp
+ description: Age
+ name: Age
+ type: date
+ group: configuration.konghq.com
+ names:
+ kind: KongConsumer
+ plural: kongconsumers
+ shortNames:
+ - kc
+ scope: Namespaced
+ subresources:
+ status: {}
+ validation:
+ openAPIV3Schema:
+ properties:
+ credentials:
+ items:
+ type: string
+ type: array
+ custom_id:
+ type: string
+ username:
+ type: string
+ version: v1
+---
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: kongingresses.configuration.konghq.com
+spec:
+ group: configuration.konghq.com
+ names:
+ kind: KongIngress
+ plural: kongingresses
+ shortNames:
+ - ki
+ scope: Namespaced
+ subresources:
+ status: {}
+ validation:
+ openAPIV3Schema:
+ properties:
+ proxy:
+ properties:
+ connect_timeout:
+ minimum: 0
+ type: integer
+ path:
+ pattern: ^/.*$
+ type: string
+ protocol:
+ enum:
+ - http
+ - https
+ - grpc
+ - grpcs
+ - tcp
+ - tls
+ type: string
+ read_timeout:
+ minimum: 0
+ type: integer
+ retries:
+ minimum: 0
+ type: integer
+ write_timeout:
+ minimum: 0
+ type: integer
+ type: object
+ route:
+ properties:
+ headers:
+ additionalProperties:
+ items:
+ type: string
+ type: array
+ type: object
+ https_redirect_status_code:
+ type: integer
+ methods:
+ items:
+ type: string
+ type: array
+ path_handling:
+ enum:
+ - v0
+ - v1
+ type: string
+ preserve_host:
+ type: boolean
+ protocols:
+ items:
+ enum:
+ - http
+ - https
+ - grpc
+ - grpcs
+ - tcp
+ - tls
+ type: string
+ type: array
+ regex_priority:
+ type: integer
+ request_buffering:
+ type: boolean
+ response_buffering:
+ type: boolean
+ snis:
+ items:
+ type: string
+ type: array
+ strip_path:
+ type: boolean
+ upstream:
+ properties:
+ algorithm:
+ enum:
+ - round-robin
+ - consistent-hashing
+ - least-connections
+ type: string
+ hash_fallback:
+ type: string
+ hash_fallback_header:
+ type: string
+ hash_on:
+ type: string
+ hash_on_cookie:
+ type: string
+ hash_on_cookie_path:
+ type: string
+ hash_on_header:
+ type: string
+ healthchecks:
+ properties:
+ active:
+ properties:
+ concurrency:
+ minimum: 1
+ type: integer
+ healthy:
+ properties:
+ http_statuses:
+ items:
+ type: integer
+ type: array
+ interval:
+ minimum: 0
+ type: integer
+ successes:
+ minimum: 0
+ type: integer
+ type: object
+ http_path:
+ pattern: ^/.*$
+ type: string
+ timeout:
+ minimum: 0
+ type: integer
+ unhealthy:
+ properties:
+ http_failures:
+ minimum: 0
+ type: integer
+ http_statuses:
+ items:
+ type: integer
+ type: array
+ interval:
+ minimum: 0
+ type: integer
+ tcp_failures:
+ minimum: 0
+ type: integer
+ timeout:
+ minimum: 0
+ type: integer
+ type: object
+ type: object
+ passive:
+ properties:
+ healthy:
+ properties:
+ http_statuses:
+ items:
+ type: integer
+ type: array
+ interval:
+ minimum: 0
+ type: integer
+ successes:
+ minimum: 0
+ type: integer
+ type: object
+ unhealthy:
+ properties:
+ http_failures:
+ minimum: 0
+ type: integer
+ http_statuses:
+ items:
+ type: integer
+ type: array
+ interval:
+ minimum: 0
+ type: integer
+ tcp_failures:
+ minimum: 0
+ type: integer
+ timeout:
+ minimum: 0
+ type: integer
+ type: object
+ type: object
+ threshold:
+ type: integer
+ type: object
+ host_header:
+ type: string
+ slots:
+ minimum: 10
+ type: integer
+ type: object
+ version: v1
+---
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: kongplugins.configuration.konghq.com
+spec:
+ additionalPrinterColumns:
+ - JSONPath: .plugin
+ description: Name of the plugin
+ name: Plugin-Type
+ type: string
+ - JSONPath: .metadata.creationTimestamp
+ description: Age
+ name: Age
+ type: date
+ - JSONPath: .disabled
+ description: Indicates if the plugin is disabled
+ name: Disabled
+ priority: 1
+ type: boolean
+ - JSONPath: .config
+ description: Configuration of the plugin
+ name: Config
+ priority: 1
+ type: string
+ group: configuration.konghq.com
+ names:
+ kind: KongPlugin
+ plural: kongplugins
+ shortNames:
+ - kp
+ scope: Namespaced
+ subresources:
+ status: {}
+ validation:
+ openAPIV3Schema:
+ properties:
+ config:
+ type: object
+ configFrom:
+ properties:
+ secretKeyRef:
+ properties:
+ key:
+ type: string
+ name:
+ type: string
+ required:
+ - name
+ - key
+ type: object
+ type: object
+ disabled:
+ type: boolean
+ plugin:
+ type: string
+ protocols:
+ items:
+ enum:
+ - http
+ - https
+ - grpc
+ - grpcs
+ - tcp
+ - tls
+ type: string
+ type: array
+ run_on:
+ enum:
+ - first
+ - second
+ - all
+ type: string
+ required:
+ - plugin
+ version: v1
+---
+apiVersion: apiextensions.k8s.io/v1beta1
+kind: CustomResourceDefinition
+metadata:
+ name: tcpingresses.configuration.konghq.com
+spec:
+ additionalPrinterColumns:
+ - JSONPath: .status.loadBalancer.ingress[*].ip
+ description: Address of the load balancer
+ name: Address
+ type: string
+ - JSONPath: .metadata.creationTimestamp
+ description: Age
+ name: Age
+ type: date
+ group: configuration.konghq.com
+ names:
+ kind: TCPIngress
+ plural: tcpingresses
+ scope: Namespaced
+ subresources:
+ status: {}
+ validation:
+ openAPIV3Schema:
+ properties:
+ apiVersion:
+ type: string
+ kind:
+ type: string
+ metadata:
+ type: object
+ spec:
+ properties:
+ rules:
+ items:
+ properties:
+ backend:
+ properties:
+ serviceName:
+ type: string
+ servicePort:
+ format: int32
+ type: integer
+ type: object
+ host:
+ type: string
+ port:
+ format: int32
+ type: integer
+ type: object
+ type: array
+ tls:
+ items:
+ properties:
+ hosts:
+ items:
+ type: string
+ type: array
+ secretName:
+ type: string
+ type: object
+ type: array
+ type: object
+ status:
+ type: object
+ version: v1beta1
+status:
+ acceptedNames:
+ kind: ""
+ plural: ""
+ conditions: []
+ storedVersions: []
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: kong-serviceaccount
+ namespace: kong
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+ name: kong-ingress-clusterrole
+rules:
+- apiGroups:
+ - ""
+ resources:
+ - endpoints
+ - nodes
+ - pods
+ - secrets
+ verbs:
+ - list
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - nodes
+ verbs:
+ - get
+- apiGroups:
+ - ""
+ resources:
+ - services
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - networking.k8s.io
+ - extensions
+ - networking.internal.knative.dev
+ resources:
+ - ingresses
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - events
+ verbs:
+ - create
+ - patch
+- apiGroups:
+ - networking.k8s.io
+ - extensions
+ - networking.internal.knative.dev
+ resources:
+ - ingresses/status
+ verbs:
+ - update
+- apiGroups:
+ - configuration.konghq.com
+ resources:
+ - tcpingresses/status
+ verbs:
+ - update
+- apiGroups:
+ - configuration.konghq.com
+ resources:
+ - kongplugins
+ - kongclusterplugins
+ - kongcredentials
+ - kongconsumers
+ - kongingresses
+ - tcpingresses
+ verbs:
+ - get
+ - list
+ - watch
+- apiGroups:
+ - ""
+ resources:
+ - configmaps
+ verbs:
+ - create
+ - get
+ - update
+---
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+ name: kong-ingress-clusterrole-nisa-binding
+roleRef:
+ apiGroup: rbac.authorization.k8s.io
+ kind: ClusterRole
+ name: kong-ingress-clusterrole
+subjects:
+- kind: ServiceAccount
+ name: kong-serviceaccount
+ namespace: kong
+---
+apiVersion: v1
+kind: Service
+metadata:
+ annotations:
+ service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
+ service.beta.kubernetes.io/aws-load-balancer-type: nlb
+ name: kong-proxy
+ namespace: kong
+spec:
+ ports:
+ - name: proxy
+ port: 80
+ protocol: TCP
+ targetPort: 8000
+ - name: proxy-ssl
+ port: 443
+ protocol: TCP
+ targetPort: 8443
+ selector:
+ app: ingress-kong
+ type: NodePort
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: kong-validation-webhook
+ namespace: kong
+spec:
+ ports:
+ - name: webhook
+ port: 443
+ protocol: TCP
+ targetPort: 8080
+ selector:
+ app: ingress-kong
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ labels:
+ app: ingress-kong
+ name: ingress-kong
+ namespace: kong
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: ingress-kong
+ template:
+ metadata:
+ annotations:
+ kuma.io/gateway: enabled
+ prometheus.io/port: "8100"
+ prometheus.io/scrape: "true"
+ traffic.sidecar.istio.io/includeInboundPorts: ""
+ labels:
+ app: ingress-kong
+ spec:
+ containers:
+ - env:
+ - name: KONG_PROXY_LISTEN
+ value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
+ - name: KONG_PORT_MAPS
+ value: 80:8000, 443:8443
+ - name: KONG_ADMIN_LISTEN
+ value: 127.0.0.1:8444 ssl
+ - name: KONG_STATUS_LISTEN
+ value: 0.0.0.0:8100
+ - name: KONG_DATABASE
+ value: "off"
+ - name: KONG_NGINX_WORKER_PROCESSES
+ value: "2"
+ - name: KONG_ADMIN_ACCESS_LOG
+ value: /dev/stdout
+ - name: KONG_ADMIN_ERROR_LOG
+ value: /dev/stderr
+ - name: KONG_PROXY_ERROR_LOG
+ value: /dev/stderr
+ image: registry.baidubce.com/serving_gateway/kong:paddle
+ lifecycle:
+ preStop:
+ exec:
+ command:
+ - /bin/sh
+ - -c
+ - kong quit
+ livenessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /status
+ port: 8100
+ scheme: HTTP
+ initialDelaySeconds: 5
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 1
+ name: proxy
+ ports:
+ - containerPort: 8000
+ name: proxy
+ protocol: TCP
+ - containerPort: 8443
+ name: proxy-ssl
+ protocol: TCP
+ - containerPort: 8100
+ name: metrics
+ protocol: TCP
+ readinessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /status
+ port: 8100
+ scheme: HTTP
+ initialDelaySeconds: 5
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 1
+ - env:
+ - name: CONTROLLER_KONG_ADMIN_URL
+ value: https://127.0.0.1:8444
+ - name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY
+ value: "true"
+ - name: CONTROLLER_PUBLISH_SERVICE
+ value: kong/kong-proxy
+ - name: POD_NAME
+ valueFrom:
+ fieldRef:
+ apiVersion: v1
+ fieldPath: metadata.name
+ - name: POD_NAMESPACE
+ valueFrom:
+ fieldRef:
+ apiVersion: v1
+ fieldPath: metadata.namespace
+ image: kong/kubernetes-ingress-controller:1.2
+ imagePullPolicy: IfNotPresent
+ livenessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /healthz
+ port: 10254
+ scheme: HTTP
+ initialDelaySeconds: 5
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 1
+ name: ingress-controller
+ ports:
+ - containerPort: 8080
+ name: webhook
+ protocol: TCP
+ readinessProbe:
+ failureThreshold: 3
+ httpGet:
+ path: /healthz
+ port: 10254
+ scheme: HTTP
+ initialDelaySeconds: 5
+ periodSeconds: 10
+ successThreshold: 1
+ timeoutSeconds: 1
+ serviceAccountName: kong-serviceaccount
diff --git a/tools/auth/serving-demo-k8s.yaml b/tools/auth/serving-demo-k8s.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..173b601aee65c295260b944bf147c9134a24206b
--- /dev/null
+++ b/tools/auth/serving-demo-k8s.yaml
@@ -0,0 +1,63 @@
+apiVersion: v1
+kind: Service
+metadata:
+ labels:
+ app: uci
+ name: uci
+spec:
+ ports:
+ - port: 9393
+ name: http
+ protocol: TCP
+ targetPort: 9393
+ selector:
+ app: uci
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ labels:
+ app: uci
+ name: uci
+spec:
+ replicas: 1
+ selector:
+ matchLabels:
+ app: uci
+ strategy: {}
+ template:
+ metadata:
+ creationTimestamp: null
+ labels:
+ app: uci
+ spec:
+ containers:
+ - image: registry.baidubce.com/serving_dev/fit_a_line:security
+ name: uci
+ imagePullPolicy: Always
+ ports:
+ - containerPort: 9393
+ workingDir: /home/fit_a_line/
+ name: uci
+ command: ['/bin/bash', '-c']
+ args: ["python3.6 -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9393 --name uci"]
+ env:
+ - name: SERVING_BIN
+ value: "/usr/local/serving_bin/serving"
+ - name: NODE_NAME
+ valueFrom:
+ fieldRef:
+ fieldPath: spec.nodeName
+ - name: POD_NAME
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.name
+ - name: POD_NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ resources: {}