Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
PaddlePaddle
Serving
提交
6830973d
S
Serving
项目概览
PaddlePaddle
/
Serving
12 个月 前同步成功
通知
183
Star
833
Fork
253
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
105
列表
看板
标记
里程碑
合并请求
10
Wiki
2
Wiki
分析
仓库
DevOps
项目成员
Pages
S
Serving
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
105
Issue
105
列表
看板
标记
里程碑
合并请求
10
合并请求
10
Pages
分析
分析
仓库分析
DevOps
Wiki
2
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
6830973d
编写于
12月 22, 2020
作者:
J
Jiawei Wang
提交者:
GitHub
12月 22, 2020
浏览文件
操作
浏览文件
下载
差异文件
Merge pull request #936 from wangjiawei04/readme_0.4
Fix Readme 0.4
上级
702dc165
38dc5701
变更
26
隐藏空白更改
内联
并排
Showing
26 changed file
with
21 addition
and
1040 deletion
+21
-1040
README.md
README.md
+2
-10
README_CN.md
README_CN.md
+1
-1
doc/BERT_10_MINS.md
doc/BERT_10_MINS.md
+9
-5
doc/BERT_10_MINS_CN.md
doc/BERT_10_MINS_CN.md
+9
-4
python/examples/criteo_ctr_with_cube/README.md
python/examples/criteo_ctr_with_cube/README.md
+0
-72
python/examples/criteo_ctr_with_cube/README_CN.md
python/examples/criteo_ctr_with_cube/README_CN.md
+0
-70
python/examples/criteo_ctr_with_cube/args.py
python/examples/criteo_ctr_with_cube/args.py
+0
-105
python/examples/criteo_ctr_with_cube/benchmark.py
python/examples/criteo_ctr_with_cube/benchmark.py
+0
-91
python/examples/criteo_ctr_with_cube/benchmark.sh
python/examples/criteo_ctr_with_cube/benchmark.sh
+0
-32
python/examples/criteo_ctr_with_cube/benchmark_cube.sh
python/examples/criteo_ctr_with_cube/benchmark_cube.sh
+0
-32
python/examples/criteo_ctr_with_cube/clean.sh
python/examples/criteo_ctr_with_cube/clean.sh
+0
-4
python/examples/criteo_ctr_with_cube/criteo.py
python/examples/criteo_ctr_with_cube/criteo.py
+0
-81
python/examples/criteo_ctr_with_cube/criteo_reader.py
python/examples/criteo_ctr_with_cube/criteo_reader.py
+0
-83
python/examples/criteo_ctr_with_cube/cube/conf/cube.conf
python/examples/criteo_ctr_with_cube/cube/conf/cube.conf
+0
-13
python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf
python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf
+0
-4
python/examples/criteo_ctr_with_cube/cube/keys
python/examples/criteo_ctr_with_cube/cube/keys
+0
-10
python/examples/criteo_ctr_with_cube/cube_prepare.sh
python/examples/criteo_ctr_with_cube/cube_prepare.sh
+0
-20
python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh
python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh
+0
-22
python/examples/criteo_ctr_with_cube/gen_key.py
python/examples/criteo_ctr_with_cube/gen_key.py
+0
-20
python/examples/criteo_ctr_with_cube/get_data.sh
python/examples/criteo_ctr_with_cube/get_data.sh
+0
-2
python/examples/criteo_ctr_with_cube/local_train.py
python/examples/criteo_ctr_with_cube/local_train.py
+0
-100
python/examples/criteo_ctr_with_cube/network_conf.py
python/examples/criteo_ctr_with_cube/network_conf.py
+0
-77
python/examples/criteo_ctr_with_cube/test_client.py
python/examples/criteo_ctr_with_cube/test_client.py
+0
-59
python/examples/criteo_ctr_with_cube/test_server.py
python/examples/criteo_ctr_with_cube/test_server.py
+0
-41
python/examples/criteo_ctr_with_cube/test_server_gpu.py
python/examples/criteo_ctr_with_cube/test_server_gpu.py
+0
-41
python/examples/criteo_ctr_with_cube/test_server_quant.py
python/examples/criteo_ctr_with_cube/test_server_quant.py
+0
-41
未找到文件。
README.md
浏览文件 @
6830973d
...
...
@@ -66,15 +66,6 @@ For **Windows Users**, please read the document [Paddle Serving for Windows User
<h2
align=
"center"
>
Pre-built services with Paddle Serving
</h2>
<h3
align=
"center"
>
Latest release
</h4>
<p
align=
"center"
>
<a
href=
"https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/ocr"
>
Optical Character Recognition
</a>
<br>
<a
href=
"https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/faster_rcnn_model"
>
Object Detection
</a>
<br>
<a
href=
"https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/deeplabv3"
>
Image Segmentation
</a>
<p>
<h3
align=
"center"
>
Chinese Word Segmentation
</h4>
```
shell
...
...
@@ -133,7 +124,8 @@ python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --po
|
`use_trt`
(Only for trt version) | - | - | Run inference with TensorRT |
</center>
```
python
```
python
# A user can visit rpc service through paddle_serving_client API
from
paddle_serving_client
import
Client
import
numpy
as
np
...
...
README_CN.md
浏览文件 @
6830973d
...
...
@@ -148,7 +148,7 @@ print(fetch_map)
在这里,
`client.predict`
函数具有两个参数。
`feed`
是带有模型输入变量别名和值的
`python dict`
。
`fetch`
被要从服务器返回的预测变量赋值。 在该示例中,在训练过程中保存可服务模型时,被赋值的tensor名为
`"x"`
和
`"price"`
。
<h3
align=
"center"
>
HTTP服务
</h3>
用户也可以将数据格式处理逻辑放在服务器端进行,这样就可以直接用curl去访问服务,参考如下案例,在目录
`
`python/examples/fit_a_line`
`
用户也可以将数据格式处理逻辑放在服务器端进行,这样就可以直接用curl去访问服务,参考如下案例,在目录
`
python/examples/fit_a_line
`
```
python
from
paddle_serving_server.web_service
import
WebService
...
...
doc/BERT_10_MINS.md
浏览文件 @
6830973d
...
...
@@ -56,21 +56,25 @@ the script of client side bert_client.py is as follow:
[
//file
]:
#bert_client.py
```
python
import
os
import
sys
from
paddle_serving_client
import
Client
from
paddle_serving_client.utils
import
benchmark_args
from
paddle_serving_app.reader
import
ChineseBertReader
import
numpy
as
np
args
=
benchmark_args
()
reader
=
ChineseBertReader
()
reader
=
ChineseBertReader
(
{
"max_seq_len"
:
128
}
)
fetch
=
[
"pooled_output"
]
endpoint_list
=
[
"127.0.0.1:9292"
]
endpoint_list
=
[
'127.0.0.1:9292'
]
client
=
Client
()
client
.
load_client_config
(
"bert_seq20_client/serving_client_conf.prototxt"
)
client
.
load_client_config
(
args
.
model
)
client
.
connect
(
endpoint_list
)
for
line
in
sys
.
stdin
:
feed_dict
=
reader
.
process
(
line
)
result
=
client
.
predict
(
feed
=
feed_dict
,
fetch
=
fetch
)
for
key
in
feed_dict
.
keys
():
feed_dict
[
key
]
=
np
.
array
(
feed_dict
[
key
]).
reshape
((
128
,
1
))
result
=
client
.
predict
(
feed
=
feed_dict
,
fetch
=
fetch
,
batch
=
False
)
```
run
...
...
doc/BERT_10_MINS_CN.md
浏览文件 @
6830973d
...
...
@@ -52,18 +52,23 @@ pip install paddle_serving_app
```
python
import
sys
from
paddle_serving_client
import
Client
from
paddle_serving_client.utils
import
benchmark_args
from
paddle_serving_app.reader
import
ChineseBertReader
import
numpy
as
np
args
=
benchmark_args
()
reader
=
ChineseBertReader
()
reader
=
ChineseBertReader
(
{
"max_seq_len"
:
128
}
)
fetch
=
[
"pooled_output"
]
endpoint_list
=
[
"127.0.0.1:9292"
]
endpoint_list
=
[
'127.0.0.1:9292'
]
client
=
Client
()
client
.
load_client_config
(
"bert_seq20_client/serving_client_conf.prototxt"
)
client
.
load_client_config
(
args
.
model
)
client
.
connect
(
endpoint_list
)
for
line
in
sys
.
stdin
:
feed_dict
=
reader
.
process
(
line
)
result
=
client
.
predict
(
feed
=
feed_dict
,
fetch
=
fetch
)
for
key
in
feed_dict
.
keys
():
feed_dict
[
key
]
=
np
.
array
(
feed_dict
[
key
]).
reshape
((
128
,
1
))
result
=
client
.
predict
(
feed
=
feed_dict
,
fetch
=
fetch
,
batch
=
False
)
```
执行
...
...
python/examples/criteo_ctr_with_cube/README.md
已删除
100755 → 0
浏览文件 @
702dc165
## Criteo CTR with Sparse Parameter Indexing Service
(
[
简体中文
](
./README_CN.md
)
|English)
### Get Sample Dataset
go to directory
`python/examples/criteo_ctr_with_cube`
```
sh get_data.sh
```
### Download Model and Sparse Parameter Sequence Files
```
wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz
tar xf ctr_cube_unittest.tar.gz
mv models/ctr_client_conf ./
mv models/ctr_serving_model_kv ./
mv models/data ./cube/
```
the model will be in ./ctr_server_model_kv and ./ctr_client_config.
### Start Sparse Parameter Indexing Service
```
wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz
tar xf cube_app.tar.gz
mv cube_app/cube* ./cube/
sh cube_prepare.sh &
```
Here, the sparse parameter is loaded by cube sparse parameter indexing service Cube.
### Start RPC Predictor, the number of serving thread is 4(configurable in test_server.py)
```
python test_server.py ctr_serving_model_kv
```
### Run Prediction
```
python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data
```
### Benchmark
CPU :Intel(R) Xeon(R) CPU 6148 @ 2.40GHz
Model :
[
Criteo CTR
](
https://github.com/PaddlePaddle/Serving/blob/develop/python/examples/criteo_ctr_with_cube/network_conf.py
)
server core/thread num : 4/8
Run
```
bash benchmark.sh
```
1000 batches will be sent by every client
| client thread num | prepro | client infer | op0 | op1 | op2 | postpro | avg_latency | qps |
| ------------------ | ------ | ------------ | ------ | ----- | ------ | ------- | ----- | ----- |
| 1 | 0.035 | 1.596 | 0.021 | 0.518 | 0.0024 | 0.0025 | 6.774 | 147.7 |
| 2 | 0.034 | 1.780 | 0.027 | 0.463 | 0.0020 | 0.0023 | 6.931 | 288.3 |
| 4 | 0.038 | 2.954 | 0.025 | 0.455 | 0.0019 | 0.0027 | 8.378 | 477.5 |
| 8 | 0.044 | 8.230 | 0.028 | 0.464 | 0.0023 | 0.0034 | 14.191 | 563.8 |
| 16 | 0.048 | 21.037 | 0.028 | 0.455 | 0.0025 | 0.0041 | 27.236 | 587.5 |
the average latency of threads
![
avg cost
](
../../../doc/criteo-cube-benchmark-avgcost.png
)
The QPS is
![
qps
](
../../../doc/criteo-cube-benchmark-qps.png
)
python/examples/criteo_ctr_with_cube/README_CN.md
已删除
100644 → 0
浏览文件 @
702dc165
## 带稀疏参数索引服务的CTR预测服务
(简体中文|
[
English
](
./README.md
)
)
### 获取样例数据
进入目录
`python/examples/criteo_ctr_with_cube`
```
sh get_data.sh
```
### 下载模型和稀疏参数序列文件
```
wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz
tar xf ctr_cube_unittest.tar.gz
mv models/ctr_client_conf ./
mv models/ctr_serving_model_kv ./
mv models/data ./cube/
```
执行脚本后会在当前目录有ctr_server_model_kv和ctr_client_config文件夹。
### 启动稀疏参数索引服务
```
wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz
tar xf cube_app.tar.gz
mv cube_app/cube* ./cube/
sh cube_prepare.sh &
```
此处,模型当中的稀疏参数会被存放在稀疏参数索引服务Cube当中。
### 启动RPC预测服务,服务端线程数为4(可在test_server.py配置)
```
python test_server.py ctr_serving_model_kv
```
### 执行预测
```
python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data
```
### Benchmark
设备 :Intel(R) Xeon(R) CPU 6148 @ 2.40GHz
模型 :
[
Criteo CTR
](
https://github.com/PaddlePaddle/Serving/blob/develop/python/examples/criteo_ctr_with_cube/network_conf.py
)
server core/thread num : 4/8
执行
```
bash benchmark.sh
```
客户端每个线程会发送1000个batch
| client thread num | prepro | client infer | op0 | op1 | op2 | postpro | avg_latency | qps |
| ------------------ | ------ | ------------ | ------ | ----- | ------ | ------- | ----- | ----- |
| 1 | 0.035 | 1.596 | 0.021 | 0.518 | 0.0024 | 0.0025 | 6.774 | 147.7 |
| 2 | 0.034 | 1.780 | 0.027 | 0.463 | 0.0020 | 0.0023 | 6.931 | 288.3 |
| 4 | 0.038 | 2.954 | 0.025 | 0.455 | 0.0019 | 0.0027 | 8.378 | 477.5 |
| 8 | 0.044 | 8.230 | 0.028 | 0.464 | 0.0023 | 0.0034 | 14.191 | 563.8 |
| 16 | 0.048 | 21.037 | 0.028 | 0.455 | 0.0025 | 0.0041 | 27.236 | 587.5 |
平均每个线程耗时图如下
![
avg cost
](
../../../doc/criteo-cube-benchmark-avgcost.png
)
每个线程QPS耗时如下
![
qps
](
../../../doc/criteo-cube-benchmark-qps.png
)
python/examples/criteo_ctr_with_cube/args.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
argparse
def
parse_args
():
parser
=
argparse
.
ArgumentParser
(
description
=
"PaddlePaddle CTR example"
)
parser
.
add_argument
(
'--train_data_path'
,
type
=
str
,
default
=
'./data/raw/train.txt'
,
help
=
"The path of training dataset"
)
parser
.
add_argument
(
'--sparse_only'
,
type
=
bool
,
default
=
False
,
help
=
"Whether we use sparse features only"
)
parser
.
add_argument
(
'--test_data_path'
,
type
=
str
,
default
=
'./data/raw/valid.txt'
,
help
=
"The path of testing dataset"
)
parser
.
add_argument
(
'--batch_size'
,
type
=
int
,
default
=
1000
,
help
=
"The size of mini-batch (default:1000)"
)
parser
.
add_argument
(
'--embedding_size'
,
type
=
int
,
default
=
10
,
help
=
"The size for embedding layer (default:10)"
)
parser
.
add_argument
(
'--num_passes'
,
type
=
int
,
default
=
10
,
help
=
"The number of passes to train (default: 10)"
)
parser
.
add_argument
(
'--model_output_dir'
,
type
=
str
,
default
=
'models'
,
help
=
'The path for model to store (default: models)'
)
parser
.
add_argument
(
'--sparse_feature_dim'
,
type
=
int
,
default
=
1000001
,
help
=
'sparse feature hashing space for index processing'
)
parser
.
add_argument
(
'--is_local'
,
type
=
int
,
default
=
1
,
help
=
'Local train or distributed train (default: 1)'
)
parser
.
add_argument
(
'--cloud_train'
,
type
=
int
,
default
=
0
,
help
=
'Local train or distributed train on paddlecloud (default: 0)'
)
parser
.
add_argument
(
'--async_mode'
,
action
=
'store_true'
,
default
=
False
,
help
=
'Whether start pserver in async mode to support ASGD'
)
parser
.
add_argument
(
'--no_split_var'
,
action
=
'store_true'
,
default
=
False
,
help
=
'Whether split variables into blocks when update_method is pserver'
)
parser
.
add_argument
(
'--role'
,
type
=
str
,
default
=
'pserver'
,
# trainer or pserver
help
=
'The path for model to store (default: models)'
)
parser
.
add_argument
(
'--endpoints'
,
type
=
str
,
default
=
'127.0.0.1:6000'
,
help
=
'The pserver endpoints, like: 127.0.0.1:6000,127.0.0.1:6001'
)
parser
.
add_argument
(
'--current_endpoint'
,
type
=
str
,
default
=
'127.0.0.1:6000'
,
help
=
'The path for model to store (default: 127.0.0.1:6000)'
)
parser
.
add_argument
(
'--trainer_id'
,
type
=
int
,
default
=
0
,
help
=
'The path for model to store (default: models)'
)
parser
.
add_argument
(
'--trainers'
,
type
=
int
,
default
=
1
,
help
=
'The num of trianers, (default: 1)'
)
return
parser
.
parse_args
()
python/examples/criteo_ctr_with_cube/benchmark.py
已删除
100755 → 0
浏览文件 @
702dc165
# -*- coding: utf-8 -*-
#
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
from
paddle_serving_client
import
Client
import
sys
import
os
import
criteo
as
criteo
import
time
from
paddle_serving_client.utils
import
MultiThreadRunner
from
paddle_serving_client.utils
import
benchmark_args
from
paddle_serving_client.metric
import
auc
py_version
=
sys
.
version_info
[
0
]
args
=
benchmark_args
()
def
single_func
(
idx
,
resource
):
client
=
Client
()
print
([
resource
[
"endpoint"
][
idx
%
len
(
resource
[
"endpoint"
])]])
client
.
load_client_config
(
'ctr_client_conf/serving_client_conf.prototxt'
)
client
.
connect
([
'127.0.0.1:9292'
])
batch
=
1
buf_size
=
100
dataset
=
criteo
.
CriteoDataset
()
dataset
.
setup
(
1000001
)
test_filelists
=
[
"./raw_data/part-%d"
%
x
for
x
in
range
(
len
(
os
.
listdir
(
"./raw_data"
)))
]
reader
=
dataset
.
infer_reader
(
test_filelists
[
len
(
test_filelists
)
-
40
:],
batch
,
buf_size
)
if
args
.
request
==
"rpc"
:
fetch
=
[
"prob"
]
start
=
time
.
time
()
itr
=
1000
for
ei
in
range
(
itr
):
if
args
.
batch_size
>
0
:
feed_batch
=
[]
for
bi
in
range
(
args
.
batch_size
):
if
py_version
==
2
:
data
=
reader
().
next
()
else
:
data
=
reader
().
__next__
()
feed_dict
=
{}
feed_dict
[
'dense_input'
]
=
data
[
0
][
0
]
for
i
in
range
(
1
,
27
):
feed_dict
[
"embedding_{}.tmp_0"
.
format
(
i
-
1
)]
=
data
[
0
][
i
]
feed_batch
.
append
(
feed_dict
)
result
=
client
.
predict
(
feed
=
feed_batch
,
fetch
=
fetch
)
else
:
print
(
"unsupport batch size {}"
.
format
(
args
.
batch_size
))
elif
args
.
request
==
"http"
:
raise
(
"Not support http service."
)
end
=
time
.
time
()
qps
=
itr
*
args
.
batch_size
/
(
end
-
start
)
return
[[
end
-
start
,
qps
]]
if
__name__
==
'__main__'
:
multi_thread_runner
=
MultiThreadRunner
()
endpoint_list
=
[
"127.0.0.1:9292"
]
#result = single_func(0, {"endpoint": endpoint_list})
start
=
time
.
time
()
result
=
multi_thread_runner
.
run
(
single_func
,
args
.
thread
,
{
"endpoint"
:
endpoint_list
})
end
=
time
.
time
()
total_cost
=
end
-
start
avg_cost
=
0
qps
=
0
for
i
in
range
(
args
.
thread
):
avg_cost
+=
result
[
0
][
i
*
2
+
0
]
qps
+=
result
[
0
][
i
*
2
+
1
]
avg_cost
=
avg_cost
/
args
.
thread
print
(
"total cost: {}"
.
format
(
total_cost
))
print
(
"average total cost {} s."
.
format
(
avg_cost
))
print
(
"qps {} ins/s"
.
format
(
qps
))
python/examples/criteo_ctr_with_cube/benchmark.sh
已删除
100755 → 0
浏览文件 @
702dc165
rm
profile_log
export
FLAGS_profile_client
=
1
export
FLAGS_profile_server
=
1
wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz
--no-check-certificate
tar
xf ctr_cube_unittest.tar.gz
mv
models/ctr_client_conf ./
mv
models/ctr_serving_model_kv ./
mv
models/data ./cube/
wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz
--no-check-certificate
tar
xf cube_app.tar.gz
mv
cube_app/cube
*
./cube/
sh cube_prepare.sh &
python test_server.py ctr_serving_model_kv
>
serving_log 2>&1 &
for
thread_num
in
1 4 16
do
for
batch_size
in
1 4 16 64
do
$PYTHONROOT
/bin/python benchmark.py
--thread
$thread_num
--batch_size
$batch_size
--model
serving_client_conf/serving_client_conf.prototxt
--request
rpc
>
profile 2>&1
echo
"batch size :
$batch_size
"
echo
"thread num :
$thread_num
"
echo
"========================================"
echo
"batch size :
$batch_size
"
>>
profile_log
$PYTHONROOT
/bin/python ../util/show_profile.py profile
$thread_num
>>
profile_log
tail
-n
3 profile
>>
profile_log
done
done
ps
-ef
|grep
'serving'
|grep
-v
grep
|cut
-c
9-15 | xargs
kill
-9
python/examples/criteo_ctr_with_cube/benchmark_cube.sh
已删除
100755 → 0
浏览文件 @
702dc165
rm
profile_log
#wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz --no-check-certificate
#tar xf ctr_cube_unittest.tar.gz
mv
models/ctr_client_conf ./
mv
models/ctr_serving_model_kv ./
mv
models/data ./cube/
#wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz --no-check-certificate
#tar xf cube_app.tar.gz
mv
cube_app/cube
*
./cube/
sh cube_prepare.sh &
cp
../../../build_server/core/cube/cube-api/cube-cli
.
python gen_key.py
for
thread_num
in
1 4 16 32
do
for
batch_size
in
1000
do
./cube-cli
-config_file
./cube/conf/cube.conf
-keys
key
-dict
test_dict
-thread_num
$thread_num
--batch
$batch_size
>
profile 2>&1
echo
"batch size :
$batch_size
"
echo
"thread num :
$thread_num
"
echo
"========================================"
echo
"batch size :
$batch_size
"
>>
profile_log
echo
"thread num :
$thread_num
"
>>
profile_log
tail
-n
8 profile
>>
profile_log
done
done
ps
-ef
|grep
'cube'
|grep
-v
grep
|cut
-c
9-15 | xargs
kill
-9
python/examples/criteo_ctr_with_cube/clean.sh
已删除
100755 → 0
浏览文件 @
702dc165
ps
-ef
|
grep
cube |
awk
{
'print $2'
}
| xargs
kill
-9
rm
-rf
cube/cube_data cube/data cube/log
*
cube/nohup
*
cube/output/ cube/donefile cube/input cube/monitor cube/cube-builder.INFO
ps
-ef
|
grep test
|
awk
{
'print $2'
}
| xargs
kill
-9
ps
-ef
|
grep
serving |
awk
{
'print $2'
}
| xargs
kill
-9
python/examples/criteo_ctr_with_cube/criteo.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
sys
class
CriteoDataset
(
object
):
def
setup
(
self
,
sparse_feature_dim
):
self
.
cont_min_
=
[
0
,
-
3
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
]
self
.
cont_max_
=
[
20
,
600
,
100
,
50
,
64000
,
500
,
100
,
50
,
500
,
10
,
10
,
10
,
50
]
self
.
cont_diff_
=
[
20
,
603
,
100
,
50
,
64000
,
500
,
100
,
50
,
500
,
10
,
10
,
10
,
50
]
self
.
hash_dim_
=
sparse_feature_dim
# here, training data are lines with line_index < train_idx_
self
.
train_idx_
=
41256555
self
.
continuous_range_
=
range
(
1
,
14
)
self
.
categorical_range_
=
range
(
14
,
40
)
def
_process_line
(
self
,
line
):
features
=
line
.
rstrip
(
'
\n
'
).
split
(
'
\t
'
)
dense_feature
=
[]
sparse_feature
=
[]
for
idx
in
self
.
continuous_range_
:
if
features
[
idx
]
==
''
:
dense_feature
.
append
(
0.0
)
else
:
dense_feature
.
append
((
float
(
features
[
idx
])
-
self
.
cont_min_
[
idx
-
1
])
/
\
self
.
cont_diff_
[
idx
-
1
])
for
idx
in
self
.
categorical_range_
:
sparse_feature
.
append
(
[
hash
(
str
(
idx
)
+
features
[
idx
])
%
self
.
hash_dim_
])
return
dense_feature
,
sparse_feature
,
[
int
(
features
[
0
])]
def
infer_reader
(
self
,
filelist
,
batch
,
buf_size
):
def
local_iter
():
for
fname
in
filelist
:
with
open
(
fname
.
strip
(),
"r"
)
as
fin
:
for
line
in
fin
:
dense_feature
,
sparse_feature
,
label
=
self
.
_process_line
(
line
)
#yield dense_feature, sparse_feature, label
yield
[
dense_feature
]
+
sparse_feature
+
[
label
]
import
paddle
batch_iter
=
paddle
.
batch
(
paddle
.
reader
.
shuffle
(
local_iter
,
buf_size
=
buf_size
),
batch_size
=
batch
)
return
batch_iter
def
generate_sample
(
self
,
line
):
def
data_iter
():
dense_feature
,
sparse_feature
,
label
=
self
.
_process_line
(
line
)
feature_name
=
[
"dense_input"
]
for
idx
in
self
.
categorical_range_
:
feature_name
.
append
(
"C"
+
str
(
idx
-
13
))
feature_name
.
append
(
"label"
)
yield
zip
(
feature_name
,
[
dense_feature
]
+
sparse_feature
+
[
label
])
return
data_iter
if
__name__
==
"__main__"
:
criteo_dataset
=
CriteoDataset
()
criteo_dataset
.
setup
(
int
(
sys
.
argv
[
1
]))
criteo_dataset
.
run_from_stdin
()
python/examples/criteo_ctr_with_cube/criteo_reader.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
sys
import
paddle.fluid.incubate.data_generator
as
dg
class
CriteoDataset
(
dg
.
MultiSlotDataGenerator
):
def
setup
(
self
,
sparse_feature_dim
):
self
.
cont_min_
=
[
0
,
-
3
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
,
0
]
self
.
cont_max_
=
[
20
,
600
,
100
,
50
,
64000
,
500
,
100
,
50
,
500
,
10
,
10
,
10
,
50
]
self
.
cont_diff_
=
[
20
,
603
,
100
,
50
,
64000
,
500
,
100
,
50
,
500
,
10
,
10
,
10
,
50
]
self
.
hash_dim_
=
sparse_feature_dim
# here, training data are lines with line_index < train_idx_
self
.
train_idx_
=
41256555
self
.
continuous_range_
=
range
(
1
,
14
)
self
.
categorical_range_
=
range
(
14
,
40
)
def
_process_line
(
self
,
line
):
features
=
line
.
rstrip
(
'
\n
'
).
split
(
'
\t
'
)
dense_feature
=
[]
sparse_feature
=
[]
for
idx
in
self
.
continuous_range_
:
if
features
[
idx
]
==
''
:
dense_feature
.
append
(
0.0
)
else
:
dense_feature
.
append
((
float
(
features
[
idx
])
-
self
.
cont_min_
[
idx
-
1
])
/
\
self
.
cont_diff_
[
idx
-
1
])
for
idx
in
self
.
categorical_range_
:
sparse_feature
.
append
(
[
hash
(
str
(
idx
)
+
features
[
idx
])
%
self
.
hash_dim_
])
return
dense_feature
,
sparse_feature
,
[
int
(
features
[
0
])]
def
infer_reader
(
self
,
filelist
,
batch
,
buf_size
):
def
local_iter
():
for
fname
in
filelist
:
with
open
(
fname
.
strip
(),
"r"
)
as
fin
:
for
line
in
fin
:
dense_feature
,
sparse_feature
,
label
=
self
.
_process_line
(
line
)
#yield dense_feature, sparse_feature, label
yield
[
dense_feature
]
+
sparse_feature
+
[
label
]
import
paddle
batch_iter
=
paddle
.
batch
(
paddle
.
reader
.
shuffle
(
local_iter
,
buf_size
=
buf_size
),
batch_size
=
batch
)
return
batch_iter
def
generate_sample
(
self
,
line
):
def
data_iter
():
dense_feature
,
sparse_feature
,
label
=
self
.
_process_line
(
line
)
feature_name
=
[
"dense_input"
]
for
idx
in
self
.
categorical_range_
:
feature_name
.
append
(
"C"
+
str
(
idx
-
13
))
feature_name
.
append
(
"label"
)
yield
zip
(
feature_name
,
[
dense_feature
]
+
sparse_feature
+
[
label
])
return
data_iter
if
__name__
==
"__main__"
:
criteo_dataset
=
CriteoDataset
()
criteo_dataset
.
setup
(
int
(
sys
.
argv
[
1
]))
criteo_dataset
.
run_from_stdin
()
python/examples/criteo_ctr_with_cube/cube/conf/cube.conf
已删除
100755 → 0
浏览文件 @
702dc165
[{
"dict_name"
:
"test_dict"
,
"shard"
:
1
,
"dup"
:
1
,
"timeout"
:
200
,
"retry"
:
3
,
"backup_request"
:
100
,
"type"
:
"ipport_list"
,
"load_balancer"
:
"rr"
,
"nodes"
: [{
"ipport_list"
:
"list://127.0.0.1:8027"
}]
}]
python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf
已删除
100755 → 0
浏览文件 @
702dc165
--
port
=
8027
--
dict_split
=
1
--
in_mem
=
true
--
log_dir
=./
log
/
python/examples/criteo_ctr_with_cube/cube/keys
已删除
100755 → 0
浏览文件 @
702dc165
1
2
3
4
5
6
7
8
9
10
python/examples/criteo_ctr_with_cube/cube_prepare.sh
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
#! /bin/bash
mkdir
-p
cube_model
mkdir
-p
cube/data
./cube/cube-builder
-dict_name
=
test_dict
-job_mode
=
base
-last_version
=
0
-cur_version
=
0
-depend_version
=
0
-input_path
=
./cube_model
-output_path
=
${
PWD
}
/cube/data
-shard_num
=
1
-only_build
=
false
cd
cube
&&
./cube
python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
#! /bin/bash
mkdir
-p
cube_model
mkdir
-p
cube/data
./seq_generator ctr_serving_model/SparseFeatFactors ./cube_model/feature 8
./cube/cube-builder
-dict_name
=
test_dict
-job_mode
=
base
-last_version
=
0
-cur_version
=
0
-depend_version
=
0
-input_path
=
./cube_model
-output_path
=
${
PWD
}
/cube/data
-shard_num
=
1
-only_build
=
false
mv
./cube/data/0_0/test_dict_part0/
*
./cube/data/
cd
cube
&&
./cube
python/examples/criteo_ctr_with_cube/gen_key.py
已删除
100644 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import
sys
import
random
with
open
(
"key"
,
"w"
)
as
f
:
for
i
in
range
(
1000000
):
f
.
write
(
"{}
\n
"
.
format
(
random
.
randint
(
0
,
999999
)))
python/examples/criteo_ctr_with_cube/get_data.sh
已删除
100755 → 0
浏览文件 @
702dc165
wget
--no-check-certificate
https://paddle-serving.bj.bcebos.com/data/ctr_prediction/ctr_data.tar.gz
tar
-zxvf
ctr_data.tar.gz
python/examples/criteo_ctr_with_cube/local_train.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
from
__future__
import
print_function
from
args
import
parse_args
import
os
import
paddle.fluid
as
fluid
import
sys
from
network_conf
import
dnn_model
dense_feature_dim
=
13
def
train
():
args
=
parse_args
()
sparse_only
=
args
.
sparse_only
if
not
os
.
path
.
isdir
(
args
.
model_output_dir
):
os
.
mkdir
(
args
.
model_output_dir
)
dense_input
=
fluid
.
layers
.
data
(
name
=
"dense_input"
,
shape
=
[
dense_feature_dim
],
dtype
=
'float32'
)
sparse_input_ids
=
[
fluid
.
layers
.
data
(
name
=
"C"
+
str
(
i
),
shape
=
[
1
],
lod_level
=
1
,
dtype
=
"int64"
)
for
i
in
range
(
1
,
27
)
]
label
=
fluid
.
layers
.
data
(
name
=
'label'
,
shape
=
[
1
],
dtype
=
'int64'
)
#nn_input = None if sparse_only else dense_input
nn_input
=
dense_input
predict_y
,
loss
,
auc_var
,
batch_auc_var
,
infer_vars
=
dnn_model
(
nn_input
,
sparse_input_ids
,
label
,
args
.
embedding_size
,
args
.
sparse_feature_dim
)
optimizer
=
fluid
.
optimizer
.
SGD
(
learning_rate
=
1e-4
)
optimizer
.
minimize
(
loss
)
exe
=
fluid
.
Executor
(
fluid
.
CPUPlace
())
exe
.
run
(
fluid
.
default_startup_program
())
dataset
=
fluid
.
DatasetFactory
().
create_dataset
(
"InMemoryDataset"
)
dataset
.
set_use_var
([
dense_input
]
+
sparse_input_ids
+
[
label
])
python_executable
=
"python"
pipe_command
=
"{} criteo_reader.py {}"
.
format
(
python_executable
,
args
.
sparse_feature_dim
)
dataset
.
set_pipe_command
(
pipe_command
)
dataset
.
set_batch_size
(
128
)
thread_num
=
10
dataset
.
set_thread
(
thread_num
)
whole_filelist
=
[
"raw_data/part-%d"
%
x
for
x
in
range
(
len
(
os
.
listdir
(
"raw_data"
)))
]
print
(
whole_filelist
)
dataset
.
set_filelist
(
whole_filelist
[:
100
])
dataset
.
load_into_memory
()
fluid
.
layers
.
Print
(
auc_var
)
epochs
=
1
for
i
in
range
(
epochs
):
exe
.
train_from_dataset
(
program
=
fluid
.
default_main_program
(),
dataset
=
dataset
,
debug
=
True
)
print
(
"epoch {} finished"
.
format
(
i
))
import
paddle_serving_client.io
as
server_io
feed_var_dict
=
{}
feed_var_dict
[
'dense_input'
]
=
dense_input
for
i
,
sparse
in
enumerate
(
sparse_input_ids
):
feed_var_dict
[
"embedding_{}.tmp_0"
.
format
(
i
)]
=
sparse
fetch_var_dict
=
{
"prob"
:
predict_y
}
feed_kv_dict
=
{}
feed_kv_dict
[
'dense_input'
]
=
dense_input
for
i
,
emb
in
enumerate
(
infer_vars
):
feed_kv_dict
[
"embedding_{}.tmp_0"
.
format
(
i
)]
=
emb
fetch_var_dict
=
{
"prob"
:
predict_y
}
server_io
.
save_model
(
"ctr_serving_model"
,
"ctr_client_conf"
,
feed_var_dict
,
fetch_var_dict
,
fluid
.
default_main_program
())
server_io
.
save_model
(
"ctr_serving_model_kv"
,
"ctr_client_conf_kv"
,
feed_kv_dict
,
fetch_var_dict
,
fluid
.
default_main_program
())
if
__name__
==
'__main__'
:
train
()
python/examples/criteo_ctr_with_cube/network_conf.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
paddle.fluid
as
fluid
import
math
def
dnn_model
(
dense_input
,
sparse_inputs
,
label
,
embedding_size
,
sparse_feature_dim
):
def
embedding_layer
(
input
):
emb
=
fluid
.
layers
.
embedding
(
input
=
input
,
is_sparse
=
True
,
is_distributed
=
False
,
size
=
[
sparse_feature_dim
,
embedding_size
],
param_attr
=
fluid
.
ParamAttr
(
name
=
"SparseFeatFactors"
,
initializer
=
fluid
.
initializer
.
Uniform
()))
x
=
fluid
.
layers
.
sequence_pool
(
input
=
emb
,
pool_type
=
'sum'
)
return
emb
,
x
def
mlp_input_tensor
(
emb_sums
,
dense_tensor
):
#if isinstance(dense_tensor, fluid.Variable):
# return fluid.layers.concat(emb_sums, axis=1)
#else:
return
fluid
.
layers
.
concat
(
emb_sums
+
[
dense_tensor
],
axis
=
1
)
def
mlp
(
mlp_input
):
fc1
=
fluid
.
layers
.
fc
(
input
=
mlp_input
,
size
=
400
,
act
=
'relu'
,
param_attr
=
fluid
.
ParamAttr
(
initializer
=
fluid
.
initializer
.
Normal
(
scale
=
1
/
math
.
sqrt
(
mlp_input
.
shape
[
1
]))))
fc2
=
fluid
.
layers
.
fc
(
input
=
fc1
,
size
=
400
,
act
=
'relu'
,
param_attr
=
fluid
.
ParamAttr
(
initializer
=
fluid
.
initializer
.
Normal
(
scale
=
1
/
math
.
sqrt
(
fc1
.
shape
[
1
]))))
fc3
=
fluid
.
layers
.
fc
(
input
=
fc2
,
size
=
400
,
act
=
'relu'
,
param_attr
=
fluid
.
ParamAttr
(
initializer
=
fluid
.
initializer
.
Normal
(
scale
=
1
/
math
.
sqrt
(
fc2
.
shape
[
1
]))))
pre
=
fluid
.
layers
.
fc
(
input
=
fc3
,
size
=
2
,
act
=
'softmax'
,
param_attr
=
fluid
.
ParamAttr
(
initializer
=
fluid
.
initializer
.
Normal
(
scale
=
1
/
math
.
sqrt
(
fc3
.
shape
[
1
]))))
return
pre
emb_pair_sums
=
list
(
map
(
embedding_layer
,
sparse_inputs
))
emb_sums
=
[
x
[
1
]
for
x
in
emb_pair_sums
]
infer_vars
=
[
x
[
0
]
for
x
in
emb_pair_sums
]
mlp_in
=
mlp_input_tensor
(
emb_sums
,
dense_input
)
predict
=
mlp
(
mlp_in
)
cost
=
fluid
.
layers
.
cross_entropy
(
input
=
predict
,
label
=
label
)
avg_cost
=
fluid
.
layers
.
reduce_sum
(
cost
)
accuracy
=
fluid
.
layers
.
accuracy
(
input
=
predict
,
label
=
label
)
auc_var
,
batch_auc_var
,
auc_states
=
\
fluid
.
layers
.
auc
(
input
=
predict
,
label
=
label
,
num_thresholds
=
2
**
12
,
slide_steps
=
20
)
return
predict
,
avg_cost
,
auc_var
,
batch_auc_var
,
infer_vars
python/examples/criteo_ctr_with_cube/test_client.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
from
paddle_serving_client
import
Client
import
sys
import
os
import
criteo
as
criteo
import
time
from
paddle_serving_client.metric
import
auc
import
numpy
as
np
py_version
=
sys
.
version_info
[
0
]
client
=
Client
()
client
.
load_client_config
(
sys
.
argv
[
1
])
client
.
connect
([
"127.0.0.1:9292"
])
batch
=
1
buf_size
=
100
dataset
=
criteo
.
CriteoDataset
()
dataset
.
setup
(
1000001
)
test_filelists
=
[
"{}/part-0"
.
format
(
sys
.
argv
[
2
])]
reader
=
dataset
.
infer_reader
(
test_filelists
,
batch
,
buf_size
)
label_list
=
[]
prob_list
=
[]
start
=
time
.
time
()
for
ei
in
range
(
10000
):
if
py_version
==
2
:
data
=
reader
().
next
()
else
:
data
=
reader
().
__next__
()
feed_dict
=
{}
feed_dict
[
'dense_input'
]
=
np
.
array
(
data
[
0
][
0
]).
astype
(
"float32"
).
reshape
(
1
,
13
)
feed_dict
[
'dense_input.lod'
]
=
[
0
,
1
]
for
i
in
range
(
1
,
27
):
tmp_data
=
np
.
array
(
data
[
0
][
i
]).
astype
(
np
.
int64
)
feed_dict
[
"embedding_{}.tmp_0"
.
format
(
i
-
1
)]
=
tmp_data
.
reshape
(
(
1
,
len
(
data
[
0
][
i
])))
feed_dict
[
"embedding_{}.tmp_0.lod"
.
format
(
i
-
1
)]
=
[
0
,
1
]
fetch_map
=
client
.
predict
(
feed
=
feed_dict
,
fetch
=
[
"prob"
],
batch
=
True
)
prob_list
.
append
(
fetch_map
[
'prob'
][
0
][
1
])
label_list
.
append
(
data
[
0
][
-
1
][
0
])
print
(
auc
(
label_list
,
prob_list
))
end
=
time
.
time
()
print
(
end
-
start
)
python/examples/criteo_ctr_with_cube/test_server.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
os
import
sys
from
paddle_serving_server
import
OpMaker
from
paddle_serving_server
import
OpSeqMaker
from
paddle_serving_server
import
Server
op_maker
=
OpMaker
()
read_op
=
op_maker
.
create
(
'general_reader'
)
general_dist_kv_infer_op
=
op_maker
.
create
(
'general_dist_kv_infer'
)
response_op
=
op_maker
.
create
(
'general_response'
)
op_seq_maker
=
OpSeqMaker
()
op_seq_maker
.
add_op
(
read_op
)
op_seq_maker
.
add_op
(
general_dist_kv_infer_op
)
op_seq_maker
.
add_op
(
response_op
)
server
=
Server
()
server
.
set_op_sequence
(
op_seq_maker
.
get_op_sequence
())
server
.
set_num_threads
(
4
)
server
.
load_model_config
(
sys
.
argv
[
1
])
server
.
prepare_server
(
workdir
=
"work_dir1"
,
port
=
9292
,
device
=
"cpu"
,
cube_conf
=
"./cube/conf/cube.conf"
)
server
.
run_server
()
python/examples/criteo_ctr_with_cube/test_server_gpu.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
os
import
sys
from
paddle_serving_server_gpu
import
OpMaker
from
paddle_serving_server_gpu
import
OpSeqMaker
from
paddle_serving_server_gpu
import
Server
op_maker
=
OpMaker
()
read_op
=
op_maker
.
create
(
'general_reader'
)
general_dist_kv_infer_op
=
op_maker
.
create
(
'general_dist_kv_infer'
)
response_op
=
op_maker
.
create
(
'general_response'
)
op_seq_maker
=
OpSeqMaker
()
op_seq_maker
.
add_op
(
read_op
)
op_seq_maker
.
add_op
(
general_dist_kv_infer_op
)
op_seq_maker
.
add_op
(
response_op
)
server
=
Server
()
server
.
set_op_sequence
(
op_seq_maker
.
get_op_sequence
())
server
.
set_num_threads
(
4
)
server
.
load_model_config
(
sys
.
argv
[
1
])
server
.
prepare_server
(
workdir
=
"work_dir1"
,
port
=
9292
,
device
=
"cpu"
,
cube_conf
=
"./cube/conf/cube.conf"
)
server
.
run_server
()
python/examples/criteo_ctr_with_cube/test_server_quant.py
已删除
100755 → 0
浏览文件 @
702dc165
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# pylint: disable=doc-string-missing
import
os
import
sys
from
paddle_serving_server
import
OpMaker
from
paddle_serving_server
import
OpSeqMaker
from
paddle_serving_server
import
Server
op_maker
=
OpMaker
()
read_op
=
op_maker
.
create
(
'general_reader'
)
general_dist_kv_infer_op
=
op_maker
.
create
(
'general_dist_kv_quant_infer'
)
response_op
=
op_maker
.
create
(
'general_response'
)
op_seq_maker
=
OpSeqMaker
()
op_seq_maker
.
add_op
(
read_op
)
op_seq_maker
.
add_op
(
general_dist_kv_infer_op
)
op_seq_maker
.
add_op
(
response_op
)
server
=
Server
()
server
.
set_op_sequence
(
op_seq_maker
.
get_op_sequence
())
server
.
set_num_threads
(
4
)
server
.
load_model_config
(
sys
.
argv
[
1
])
server
.
prepare_server
(
workdir
=
"work_dir1"
,
port
=
9292
,
device
=
"cpu"
,
cube_conf
=
"./cube/conf/cube.conf"
)
server
.
run_server
()
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录