diff --git a/README.md b/README.md index 7739870de71223cc29cac76e7864632fbd9adcdb..a0d46d5c1153bb90f314b572ca8e7e82946d70ff 100644 --- a/README.md +++ b/README.md @@ -66,15 +66,6 @@ For **Windows Users**, please read the document [Paddle Serving for Windows User

Pre-built services with Paddle Serving

-

Latest release

-

- Optical Character Recognition -
- Object Detection -
- Image Segmentation -

-

Chinese Word Segmentation

``` shell @@ -133,7 +124,8 @@ python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --po | `use_trt` (Only for trt version) | - | - | Run inference with TensorRT | -``` python + +```python # A user can visit rpc service through paddle_serving_client API from paddle_serving_client import Client import numpy as np @@ -147,13 +139,6 @@ print(fetch_map) ``` Here, `client.predict` function has two arguments. `feed` is a `python dict` with model input variable alias name and values. `fetch` assigns the prediction variables to be returned from servers. In the example, the name of `"x"` and `"price"` are assigned when the servable model is saved during training. -

Some Key Features of Paddle Serving

- -- Integrate with Paddle training pipeline seamlessly, most paddle models can be deployed **with one line command**. -- **Industrial serving features** supported, such as models management, online loading, online A/B testing etc. -- **Distributed Key-Value indexing** supported which is especially useful for large scale sparse features as model inputs. -- **Highly concurrent and efficient communication** between clients and servers supported. -- **Multiple programming languages** supported on client side, such as Golang, C++ and python. ### WEB service @@ -189,6 +174,14 @@ the response is {"result":{"price":[[18.901151657104492]]}} ``` +

Some Key Features of Paddle Serving

+ +- Integrate with Paddle training pipeline seamlessly, most paddle models can be deployed **with one line command**. +- **Industrial serving features** supported, such as models management, online loading, online A/B testing etc. +- **Distributed Key-Value indexing** supported which is especially useful for large scale sparse features as model inputs. +- **Highly concurrent and efficient communication** between clients and servers supported. +- **Multiple programming languages** supported on client side, such as Golang, C++ and python. +

Document

### New to Paddle Serving @@ -235,6 +228,10 @@ To connect with other users and contributors, welcome to join our [Slack channel If you want to contribute code to Paddle Serving, please reference [Contribution Guidelines](doc/CONTRIBUTE.md) +- Special Thanks to [@BeyondYourself](https://github.com/BeyondYourself) in complementing the gRPC tutorial, updating the FAQ doc and modifying the mdkir command +- Special Thanks to [@mcl-stone](https://github.com/mcl-stone) in updating faster_rcnn benchmark +- Special Thanks to [@cg82616424](https://github.com/cg82616424) in updating the unet benchmark and modifying resize comment error + ### Feedback For any feedback or to report a bug, please propose a [GitHub Issue](https://github.com/PaddlePaddle/Serving/issues). diff --git a/README_CN.md b/README_CN.md index 56dafdf1ccd1406fa1c49517115f5b40b6a4a123..571b7b00c1252093887a1b5562e03437f51837c4 100644 --- a/README_CN.md +++ b/README_CN.md @@ -148,7 +148,7 @@ print(fetch_map) 在这里，`client.predict`函数具有两个参数。 `feed`是带有模型输入变量别名和值的`python dict`。 `fetch`被要从服务器返回的预测变量赋值。 在该示例中，在训练过程中保存可服务模型时，被赋值的tensor名为`"x"`和`"price"`。

HTTP服务

-用户也可以将数据格式处理逻辑放在服务器端进行，这样就可以直接用curl去访问服务，参考如下案例，在目录``python/examples/fit_a_line`` +用户也可以将数据格式处理逻辑放在服务器端进行，这样就可以直接用curl去访问服务，参考如下案例，在目录`python/examples/fit_a_line` ```python from paddle_serving_server.web_service import WebService @@ -232,6 +232,10 @@ curl -H "Content-Type:application/json" -X POST -d '{"feed":[{"x": [0.0137, -0.1 如果您想为Paddle Serving贡献代码，请参考 [Contribution Guidelines](doc/CONTRIBUTE.md) +- 特别感谢 [@BeyondYourself](https://github.com/BeyondYourself) 提供grpc教程，更新FAQ教程，整理文件目录。 +- 特别感谢 [@mcl-stone](https://github.com/mcl-stone) 提供faster rcnn benchmark脚本 +- 特别感谢 [@cg82616424](https://github.com/cg82616424) 提供unet benchmark脚本和修改部分注释错误 + ### 反馈 如有任何反馈或是bug，请在 [GitHub Issue](https://github.com/PaddlePaddle/Serving/issues)提交 diff --git a/doc/BERT_10_MINS.md b/doc/BERT_10_MINS.md index 53e51768d3eaee6a1faac8d9ae2c62e7f1aa63ee..7b981d7eda467197d1b1b741c35b00c97b74c532 100644 --- a/doc/BERT_10_MINS.md +++ b/doc/BERT_10_MINS.md @@ -56,21 +56,25 @@ the script of client side bert_client.py is as follow: [//file]:#bert_client.py ``` python -import os import sys from paddle_serving_client import Client +from paddle_serving_client.utils import benchmark_args from paddle_serving_app.reader import ChineseBertReader +import numpy as np +args = benchmark_args() -reader = ChineseBertReader() +reader = ChineseBertReader({"max_seq_len": 128}) fetch = ["pooled_output"] -endpoint_list = ["127.0.0.1:9292"] +endpoint_list = ['127.0.0.1:9292'] client = Client() -client.load_client_config("bert_seq20_client/serving_client_conf.prototxt") +client.load_client_config(args.model) client.connect(endpoint_list) for line in sys.stdin: feed_dict = reader.process(line) - result = client.predict(feed=feed_dict, fetch=fetch) + for key in feed_dict.keys(): + feed_dict[key] = np.array(feed_dict[key]).reshape((128, 1)) + result = client.predict(feed=feed_dict, fetch=fetch, batch=False) ``` run diff --git a/doc/BERT_10_MINS_CN.md b/doc/BERT_10_MINS_CN.md index e4904d86b6a056ba74b6ed85b47745575b749279..b0578e8e84de7e36694e879e5a64737d275c505c 100644 --- a/doc/BERT_10_MINS_CN.md +++ b/doc/BERT_10_MINS_CN.md @@ -52,18 +52,23 @@ pip install paddle_serving_app ``` python import sys from paddle_serving_client import Client +from paddle_serving_client.utils import benchmark_args from paddle_serving_app.reader import ChineseBertReader +import numpy as np +args = benchmark_args() -reader = ChineseBertReader() +reader = ChineseBertReader({"max_seq_len": 128}) fetch = ["pooled_output"] -endpoint_list = ["127.0.0.1:9292"] +endpoint_list = ['127.0.0.1:9292'] client = Client() -client.load_client_config("bert_seq20_client/serving_client_conf.prototxt") +client.load_client_config(args.model) client.connect(endpoint_list) for line in sys.stdin: feed_dict = reader.process(line) - result = client.predict(feed=feed_dict, fetch=fetch) + for key in feed_dict.keys(): + feed_dict[key] = np.array(feed_dict[key]).reshape((128, 1)) + result = client.predict(feed=feed_dict, fetch=fetch, batch=False) ``` 执行 diff --git a/python/examples/criteo_ctr_with_cube/README.md b/python/examples/criteo_ctr_with_cube/README.md deleted file mode 100755 index 493b3d72c1fff9275c2a99cfee45efd4bef1af4c..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/README.md +++ /dev/null @@ -1,72 +0,0 @@ -## Criteo CTR with Sparse Parameter Indexing Service - -([简体中文](./README_CN.md)|English) - -### Get Sample Dataset - -go to directory `python/examples/criteo_ctr_with_cube` -``` -sh get_data.sh -``` - -### Download Model and Sparse Parameter Sequence Files -``` -wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz -tar xf ctr_cube_unittest.tar.gz -mv models/ctr_client_conf ./ -mv models/ctr_serving_model_kv ./ -mv models/data ./cube/ -``` -the model will be in ./ctr_server_model_kv and ./ctr_client_config. - -### Start Sparse Parameter Indexing Service -``` -wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz -tar xf cube_app.tar.gz -mv cube_app/cube* ./cube/ -sh cube_prepare.sh & -``` - -Here, the sparse parameter is loaded by cube sparse parameter indexing service Cube. - -### Start RPC Predictor, the number of serving thread is 4（configurable in test_server.py） - -``` -python test_server.py ctr_serving_model_kv -``` - -### Run Prediction - -``` -python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data -``` - -### Benchmark - -CPU ：Intel(R) Xeon(R) CPU 6148 @ 2.40GHz - -Model ：[Criteo CTR](https://github.com/PaddlePaddle/Serving/blob/develop/python/examples/criteo_ctr_with_cube/network_conf.py) - -server core/thread num ： 4/8 - -Run -``` -bash benchmark.sh -``` -1000 batches will be sent by every client - -| client thread num | prepro | client infer | op0 | op1 | op2 | postpro | avg_latency | qps | -| ------------------ | ------ | ------------ | ------ | ----- | ------ | ------- | ----- | ----- | -| 1 | 0.035 | 1.596 | 0.021 | 0.518 | 0.0024 | 0.0025 | 6.774 | 147.7 | -| 2 | 0.034 | 1.780 | 0.027 | 0.463 | 0.0020 | 0.0023 | 6.931 | 288.3 | -| 4 | 0.038 | 2.954 | 0.025 | 0.455 | 0.0019 | 0.0027 | 8.378 | 477.5 | -| 8 | 0.044 | 8.230 | 0.028 | 0.464 | 0.0023 | 0.0034 | 14.191 | 563.8 | -| 16 | 0.048 | 21.037 | 0.028 | 0.455 | 0.0025 | 0.0041 | 27.236 | 587.5 | - -the average latency of threads - -![avg cost](../../../doc/criteo-cube-benchmark-avgcost.png) - -The QPS is - -![qps](../../../doc/criteo-cube-benchmark-qps.png) diff --git a/python/examples/criteo_ctr_with_cube/README_CN.md b/python/examples/criteo_ctr_with_cube/README_CN.md deleted file mode 100644 index 7a0eb43c203aafeb38b64d249954cdabf7bf7a38..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/README_CN.md +++ /dev/null @@ -1,70 +0,0 @@ -## 带稀疏参数索引服务的CTR预测服务 -(简体中文|[English](./README.md)) - -### 获取样例数据 -进入目录 `python/examples/criteo_ctr_with_cube` -``` -sh get_data.sh -``` - -### 下载模型和稀疏参数序列文件 -``` -wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz -tar xf ctr_cube_unittest.tar.gz -mv models/ctr_client_conf ./ -mv models/ctr_serving_model_kv ./ -mv models/data ./cube/ -``` -执行脚本后会在当前目录有ctr_server_model_kv和ctr_client_config文件夹。 - -### 启动稀疏参数索引服务 -``` -wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz -tar xf cube_app.tar.gz -mv cube_app/cube* ./cube/ -sh cube_prepare.sh & -``` - -此处，模型当中的稀疏参数会被存放在稀疏参数索引服务Cube当中。 - -### 启动RPC预测服务，服务端线程数为4（可在test_server.py配置） - -``` -python test_server.py ctr_serving_model_kv -``` - -### 执行预测 - -``` -python test_client.py ctr_client_conf/serving_client_conf.prototxt ./raw_data -``` - -### Benchmark - -设备 ：Intel(R) Xeon(R) CPU 6148 @ 2.40GHz - -模型 ：[Criteo CTR](https://github.com/PaddlePaddle/Serving/blob/develop/python/examples/criteo_ctr_with_cube/network_conf.py) - -server core/thread num ： 4/8 - -执行 -``` -bash benchmark.sh -``` -客户端每个线程会发送1000个batch - -| client thread num | prepro | client infer | op0 | op1 | op2 | postpro | avg_latency | qps | -| ------------------ | ------ | ------------ | ------ | ----- | ------ | ------- | ----- | ----- | -| 1 | 0.035 | 1.596 | 0.021 | 0.518 | 0.0024 | 0.0025 | 6.774 | 147.7 | -| 2 | 0.034 | 1.780 | 0.027 | 0.463 | 0.0020 | 0.0023 | 6.931 | 288.3 | -| 4 | 0.038 | 2.954 | 0.025 | 0.455 | 0.0019 | 0.0027 | 8.378 | 477.5 | -| 8 | 0.044 | 8.230 | 0.028 | 0.464 | 0.0023 | 0.0034 | 14.191 | 563.8 | -| 16 | 0.048 | 21.037 | 0.028 | 0.455 | 0.0025 | 0.0041 | 27.236 | 587.5 | - -平均每个线程耗时图如下 - -![avg cost](../../../doc/criteo-cube-benchmark-avgcost.png) - -每个线程QPS耗时如下 - -![qps](../../../doc/criteo-cube-benchmark-qps.png) diff --git a/python/examples/criteo_ctr_with_cube/args.py b/python/examples/criteo_ctr_with_cube/args.py deleted file mode 100755 index 30124d4ebd9cd27cdb4580e654a8a47c55b178bf..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/args.py +++ /dev/null @@ -1,105 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing -import argparse - - -def parse_args(): - parser = argparse.ArgumentParser(description="PaddlePaddle CTR example") - parser.add_argument( - '--train_data_path', - type=str, - default='./data/raw/train.txt', - help="The path of training dataset") - parser.add_argument( - '--sparse_only', - type=bool, - default=False, - help="Whether we use sparse features only") - parser.add_argument( - '--test_data_path', - type=str, - default='./data/raw/valid.txt', - help="The path of testing dataset") - parser.add_argument( - '--batch_size', - type=int, - default=1000, - help="The size of mini-batch (default:1000)") - parser.add_argument( - '--embedding_size', - type=int, - default=10, - help="The size for embedding layer (default:10)") - parser.add_argument( - '--num_passes', - type=int, - default=10, - help="The number of passes to train (default: 10)") - parser.add_argument( - '--model_output_dir', - type=str, - default='models', - help='The path for model to store (default: models)') - parser.add_argument( - '--sparse_feature_dim', - type=int, - default=1000001, - help='sparse feature hashing space for index processing') - parser.add_argument( - '--is_local', - type=int, - default=1, - help='Local train or distributed train (default: 1)') - parser.add_argument( - '--cloud_train', - type=int, - default=0, - help='Local train or distributed train on paddlecloud (default: 0)') - parser.add_argument( - '--async_mode', - action='store_true', - default=False, - help='Whether start pserver in async mode to support ASGD') - parser.add_argument( - '--no_split_var', - action='store_true', - default=False, - help='Whether split variables into blocks when update_method is pserver') - parser.add_argument( - '--role', - type=str, - default='pserver', # trainer or pserver - help='The path for model to store (default: models)') - parser.add_argument( - '--endpoints', - type=str, - default='127.0.0.1:6000', - help='The pserver endpoints, like: 127.0.0.1:6000,127.0.0.1:6001') - parser.add_argument( - '--current_endpoint', - type=str, - default='127.0.0.1:6000', - help='The path for model to store (default: 127.0.0.1:6000)') - parser.add_argument( - '--trainer_id', - type=int, - default=0, - help='The path for model to store (default: models)') - parser.add_argument( - '--trainers', - type=int, - default=1, - help='The num of trianers, (default: 1)') - return parser.parse_args() diff --git a/python/examples/criteo_ctr_with_cube/benchmark.py b/python/examples/criteo_ctr_with_cube/benchmark.py deleted file mode 100755 index 324eb18e214237cdb0d228fc6b57c8efd3665cc9..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/benchmark.py +++ /dev/null @@ -1,91 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -from paddle_serving_client import Client -import sys -import os -import criteo as criteo -import time -from paddle_serving_client.utils import MultiThreadRunner -from paddle_serving_client.utils import benchmark_args -from paddle_serving_client.metric import auc - -py_version = sys.version_info[0] -args = benchmark_args() - - -def single_func(idx, resource): - client = Client() - print([resource["endpoint"][idx % len(resource["endpoint"])]]) - client.load_client_config('ctr_client_conf/serving_client_conf.prototxt') - client.connect(['127.0.0.1:9292']) - batch = 1 - buf_size = 100 - dataset = criteo.CriteoDataset() - dataset.setup(1000001) - test_filelists = [ - "./raw_data/part-%d" % x for x in range(len(os.listdir("./raw_data"))) - ] - reader = dataset.infer_reader(test_filelists[len(test_filelists) - 40:], - batch, buf_size) - if args.request == "rpc": - fetch = ["prob"] - start = time.time() - itr = 1000 - for ei in range(itr): - if args.batch_size > 0: - feed_batch = [] - for bi in range(args.batch_size): - if py_version == 2: - data = reader().next() - else: - data = reader().__next__() - feed_dict = {} - feed_dict['dense_input'] = data[0][0] - for i in range(1, 27): - feed_dict["embedding_{}.tmp_0".format(i - 1)] = data[0][ - i] - feed_batch.append(feed_dict) - result = client.predict(feed=feed_batch, fetch=fetch) - else: - print("unsupport batch size {}".format(args.batch_size)) - - elif args.request == "http": - raise ("Not support http service.") - end = time.time() - qps = itr * args.batch_size / (end - start) - return [[end - start, qps]] - - -if __name__ == '__main__': - multi_thread_runner = MultiThreadRunner() - endpoint_list = ["127.0.0.1:9292"] - #result = single_func(0, {"endpoint": endpoint_list}) - start = time.time() - result = multi_thread_runner.run(single_func, args.thread, - {"endpoint": endpoint_list}) - end = time.time() - total_cost = end - start - avg_cost = 0 - qps = 0 - for i in range(args.thread): - avg_cost += result[0][i * 2 + 0] - qps += result[0][i * 2 + 1] - avg_cost = avg_cost / args.thread - print("total cost: {}".format(total_cost)) - print("average total cost {} s.".format(avg_cost)) - print("qps {} ins/s".format(qps)) diff --git a/python/examples/criteo_ctr_with_cube/benchmark.sh b/python/examples/criteo_ctr_with_cube/benchmark.sh deleted file mode 100755 index 21daf9331ec4a7ba98ac73fc4570b024681aa06a..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/benchmark.sh +++ /dev/null @@ -1,32 +0,0 @@ -rm profile_log -export FLAGS_profile_client=1 -export FLAGS_profile_server=1 - -wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz --no-check-certificate -tar xf ctr_cube_unittest.tar.gz -mv models/ctr_client_conf ./ -mv models/ctr_serving_model_kv ./ -mv models/data ./cube/ - -wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz --no-check-certificate -tar xf cube_app.tar.gz -mv cube_app/cube* ./cube/ -sh cube_prepare.sh & - -python test_server.py ctr_serving_model_kv > serving_log 2>&1 & - -for thread_num in 1 4 16 -do -for batch_size in 1 4 16 64 -do - $PYTHONROOT/bin/python benchmark.py --thread $thread_num --batch_size $batch_size --model serving_client_conf/serving_client_conf.prototxt --request rpc > profile 2>&1 - echo "batch size : $batch_size" - echo "thread num : $thread_num" - echo "========================================" - echo "batch size : $batch_size" >> profile_log - $PYTHONROOT/bin/python ../util/show_profile.py profile $thread_num >> profile_log - tail -n 3 profile >> profile_log -done -done - -ps -ef|grep 'serving'|grep -v grep|cut -c 9-15 | xargs kill -9 diff --git a/python/examples/criteo_ctr_with_cube/benchmark_cube.sh b/python/examples/criteo_ctr_with_cube/benchmark_cube.sh deleted file mode 100755 index cb89bb1f689c54c773f7bd832567fd52db8a86e4..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/benchmark_cube.sh +++ /dev/null @@ -1,32 +0,0 @@ -rm profile_log - -#wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz --no-check-certificate -#tar xf ctr_cube_unittest.tar.gz -mv models/ctr_client_conf ./ -mv models/ctr_serving_model_kv ./ -mv models/data ./cube/ - -#wget https://paddle-serving.bj.bcebos.com/others/cube_app.tar.gz --no-check-certificate -#tar xf cube_app.tar.gz -mv cube_app/cube* ./cube/ -sh cube_prepare.sh & - -cp ../../../build_server/core/cube/cube-api/cube-cli . -python gen_key.py - -for thread_num in 1 4 16 32 -do -for batch_size in 1000 -do - ./cube-cli -config_file ./cube/conf/cube.conf -keys key -dict test_dict -thread_num $thread_num --batch $batch_size > profile 2>&1 - echo "batch size : $batch_size" - echo "thread num : $thread_num" - echo "========================================" - echo "batch size : $batch_size" >> profile_log - echo "thread num : $thread_num" >> profile_log - tail -n 8 profile >> profile_log - -done -done - -ps -ef|grep 'cube'|grep -v grep|cut -c 9-15 | xargs kill -9 diff --git a/python/examples/criteo_ctr_with_cube/clean.sh b/python/examples/criteo_ctr_with_cube/clean.sh deleted file mode 100755 index 99a4819802178f1910c5fced7d4c5a39c3037e4a..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/clean.sh +++ /dev/null @@ -1,4 +0,0 @@ -ps -ef | grep cube | awk {'print $2'} | xargs kill -9 -rm -rf cube/cube_data cube/data cube/log* cube/nohup* cube/output/ cube/donefile cube/input cube/monitor cube/cube-builder.INFO -ps -ef | grep test | awk {'print $2'} | xargs kill -9 -ps -ef | grep serving | awk {'print $2'} | xargs kill -9 diff --git a/python/examples/criteo_ctr_with_cube/criteo.py b/python/examples/criteo_ctr_with_cube/criteo.py deleted file mode 100755 index f37eb1d2c1d8df6975ec0c28923c6e17c0272290..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/criteo.py +++ /dev/null @@ -1,81 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import sys - - -class CriteoDataset(object): - def setup(self, sparse_feature_dim): - self.cont_min_ = [0, -3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] - self.cont_max_ = [ - 20, 600, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50 - ] - self.cont_diff_ = [ - 20, 603, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50 - ] - self.hash_dim_ = sparse_feature_dim - # here, training data are lines with line_index < train_idx_ - self.train_idx_ = 41256555 - self.continuous_range_ = range(1, 14) - self.categorical_range_ = range(14, 40) - - def _process_line(self, line): - features = line.rstrip('\n').split('\t') - dense_feature = [] - sparse_feature = [] - for idx in self.continuous_range_: - if features[idx] == '': - dense_feature.append(0.0) - else: - dense_feature.append((float(features[idx]) - self.cont_min_[idx - 1]) / \ - self.cont_diff_[idx - 1]) - for idx in self.categorical_range_: - sparse_feature.append( - [hash(str(idx) + features[idx]) % self.hash_dim_]) - - return dense_feature, sparse_feature, [int(features[0])] - - def infer_reader(self, filelist, batch, buf_size): - def local_iter(): - for fname in filelist: - with open(fname.strip(), "r") as fin: - for line in fin: - dense_feature, sparse_feature, label = self._process_line( - line) - #yield dense_feature, sparse_feature, label - yield [dense_feature] + sparse_feature + [label] - - import paddle - batch_iter = paddle.batch( - paddle.reader.shuffle( - local_iter, buf_size=buf_size), - batch_size=batch) - return batch_iter - - def generate_sample(self, line): - def data_iter(): - dense_feature, sparse_feature, label = self._process_line(line) - feature_name = ["dense_input"] - for idx in self.categorical_range_: - feature_name.append("C" + str(idx - 13)) - feature_name.append("label") - yield zip(feature_name, [dense_feature] + sparse_feature + [label]) - - return data_iter - - -if __name__ == "__main__": - criteo_dataset = CriteoDataset() - criteo_dataset.setup(int(sys.argv[1])) - criteo_dataset.run_from_stdin() diff --git a/python/examples/criteo_ctr_with_cube/criteo_reader.py b/python/examples/criteo_ctr_with_cube/criteo_reader.py deleted file mode 100755 index 2a80af78a9c2033bf246f703ca70a817ab786af3..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/criteo_reader.py +++ /dev/null @@ -1,83 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -import sys -import paddle.fluid.incubate.data_generator as dg - - -class CriteoDataset(dg.MultiSlotDataGenerator): - def setup(self, sparse_feature_dim): - self.cont_min_ = [0, -3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] - self.cont_max_ = [ - 20, 600, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50 - ] - self.cont_diff_ = [ - 20, 603, 100, 50, 64000, 500, 100, 50, 500, 10, 10, 10, 50 - ] - self.hash_dim_ = sparse_feature_dim - # here, training data are lines with line_index < train_idx_ - self.train_idx_ = 41256555 - self.continuous_range_ = range(1, 14) - self.categorical_range_ = range(14, 40) - - def _process_line(self, line): - features = line.rstrip('\n').split('\t') - dense_feature = [] - sparse_feature = [] - for idx in self.continuous_range_: - if features[idx] == '': - dense_feature.append(0.0) - else: - dense_feature.append((float(features[idx]) - self.cont_min_[idx - 1]) / \ - self.cont_diff_[idx - 1]) - for idx in self.categorical_range_: - sparse_feature.append( - [hash(str(idx) + features[idx]) % self.hash_dim_]) - - return dense_feature, sparse_feature, [int(features[0])] - - def infer_reader(self, filelist, batch, buf_size): - def local_iter(): - for fname in filelist: - with open(fname.strip(), "r") as fin: - for line in fin: - dense_feature, sparse_feature, label = self._process_line( - line) - #yield dense_feature, sparse_feature, label - yield [dense_feature] + sparse_feature + [label] - - import paddle - batch_iter = paddle.batch( - paddle.reader.shuffle( - local_iter, buf_size=buf_size), - batch_size=batch) - return batch_iter - - def generate_sample(self, line): - def data_iter(): - dense_feature, sparse_feature, label = self._process_line(line) - feature_name = ["dense_input"] - for idx in self.categorical_range_: - feature_name.append("C" + str(idx - 13)) - feature_name.append("label") - yield zip(feature_name, [dense_feature] + sparse_feature + [label]) - - return data_iter - - -if __name__ == "__main__": - criteo_dataset = CriteoDataset() - criteo_dataset.setup(int(sys.argv[1])) - criteo_dataset.run_from_stdin() diff --git a/python/examples/criteo_ctr_with_cube/cube/conf/cube.conf b/python/examples/criteo_ctr_with_cube/cube/conf/cube.conf deleted file mode 100755 index b70f6e34247e410f9b80054010338d3c8f452ec6..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/cube/conf/cube.conf +++ /dev/null @@ -1,13 +0,0 @@ -[{ - "dict_name": "test_dict", - "shard": 1, - "dup": 1, - "timeout": 200, - "retry": 3, - "backup_request": 100, - "type": "ipport_list", - "load_balancer": "rr", - "nodes": [{ - "ipport_list": "list://127.0.0.1:8027" - }] -}] diff --git a/python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf b/python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf deleted file mode 100755 index 21c7bddebd8f22b91d0ba26a6121007f96a4380b..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/cube/conf/gflags.conf +++ /dev/null @@ -1,4 +0,0 @@ ---port=8027 ---dict_split=1 ---in_mem=true ---log_dir=./log/ diff --git a/python/examples/criteo_ctr_with_cube/cube/keys b/python/examples/criteo_ctr_with_cube/cube/keys deleted file mode 100755 index f00c965d8307308469e537302baa73048488f162..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/cube/keys +++ /dev/null @@ -1,10 +0,0 @@ -1 -2 -3 -4 -5 -6 -7 -8 -9 -10 diff --git a/python/examples/criteo_ctr_with_cube/cube_prepare.sh b/python/examples/criteo_ctr_with_cube/cube_prepare.sh deleted file mode 100755 index 773baba4d91b02b244e766cd8ebf899cc740dbbc..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/cube_prepare.sh +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing -#! /bin/bash - -mkdir -p cube_model -mkdir -p cube/data -./cube/cube-builder -dict_name=test_dict -job_mode=base -last_version=0 -cur_version=0 -depend_version=0 -input_path=./cube_model -output_path=${PWD}/cube/data -shard_num=1 -only_build=false -cd cube && ./cube diff --git a/python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh b/python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh deleted file mode 100755 index 0db6575ab307fb81cdd0336a20bb9a8ec30d446d..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/cube_quant_prepare.sh +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing -#! /bin/bash - -mkdir -p cube_model -mkdir -p cube/data -./seq_generator ctr_serving_model/SparseFeatFactors ./cube_model/feature 8 -./cube/cube-builder -dict_name=test_dict -job_mode=base -last_version=0 -cur_version=0 -depend_version=0 -input_path=./cube_model -output_path=${PWD}/cube/data -shard_num=1 -only_build=false -mv ./cube/data/0_0/test_dict_part0/* ./cube/data/ -cd cube && ./cube diff --git a/python/examples/criteo_ctr_with_cube/gen_key.py b/python/examples/criteo_ctr_with_cube/gen_key.py deleted file mode 100644 index 115d81701fb2c8b78085c4c88a685dda992f2c27..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/gen_key.py +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import sys -import random - -with open("key", "w") as f: - for i in range(1000000): - f.write("{}\n".format(random.randint(0, 999999))) diff --git a/python/examples/criteo_ctr_with_cube/get_data.sh b/python/examples/criteo_ctr_with_cube/get_data.sh deleted file mode 100755 index 1f244b3a4aa81488bb493825576ba30c4b3bba22..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/get_data.sh +++ /dev/null @@ -1,2 +0,0 @@ -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/data/ctr_prediction/ctr_data.tar.gz -tar -zxvf ctr_data.tar.gz diff --git a/python/examples/criteo_ctr_with_cube/local_train.py b/python/examples/criteo_ctr_with_cube/local_train.py deleted file mode 100755 index d4a1bc930924e348048f7ac3e5c46381d9b6441b..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/local_train.py +++ /dev/null @@ -1,100 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -from __future__ import print_function - -from args import parse_args -import os -import paddle.fluid as fluid -import sys -from network_conf import dnn_model - -dense_feature_dim = 13 - - -def train(): - args = parse_args() - sparse_only = args.sparse_only - if not os.path.isdir(args.model_output_dir): - os.mkdir(args.model_output_dir) - dense_input = fluid.layers.data( - name="dense_input", shape=[dense_feature_dim], dtype='float32') - sparse_input_ids = [ - fluid.layers.data( - name="C" + str(i), shape=[1], lod_level=1, dtype="int64") - for i in range(1, 27) - ] - label = fluid.layers.data(name='label', shape=[1], dtype='int64') - - #nn_input = None if sparse_only else dense_input - nn_input = dense_input - predict_y, loss, auc_var, batch_auc_var, infer_vars = dnn_model( - nn_input, sparse_input_ids, label, args.embedding_size, - args.sparse_feature_dim) - - optimizer = fluid.optimizer.SGD(learning_rate=1e-4) - optimizer.minimize(loss) - - exe = fluid.Executor(fluid.CPUPlace()) - exe.run(fluid.default_startup_program()) - dataset = fluid.DatasetFactory().create_dataset("InMemoryDataset") - dataset.set_use_var([dense_input] + sparse_input_ids + [label]) - - python_executable = "python" - pipe_command = "{} criteo_reader.py {}".format(python_executable, - args.sparse_feature_dim) - - dataset.set_pipe_command(pipe_command) - dataset.set_batch_size(128) - thread_num = 10 - dataset.set_thread(thread_num) - - whole_filelist = [ - "raw_data/part-%d" % x for x in range(len(os.listdir("raw_data"))) - ] - - print(whole_filelist) - dataset.set_filelist(whole_filelist[:100]) - dataset.load_into_memory() - fluid.layers.Print(auc_var) - epochs = 1 - for i in range(epochs): - exe.train_from_dataset( - program=fluid.default_main_program(), dataset=dataset, debug=True) - print("epoch {} finished".format(i)) - - import paddle_serving_client.io as server_io - feed_var_dict = {} - feed_var_dict['dense_input'] = dense_input - for i, sparse in enumerate(sparse_input_ids): - feed_var_dict["embedding_{}.tmp_0".format(i)] = sparse - fetch_var_dict = {"prob": predict_y} - - feed_kv_dict = {} - feed_kv_dict['dense_input'] = dense_input - for i, emb in enumerate(infer_vars): - feed_kv_dict["embedding_{}.tmp_0".format(i)] = emb - fetch_var_dict = {"prob": predict_y} - - server_io.save_model("ctr_serving_model", "ctr_client_conf", feed_var_dict, - fetch_var_dict, fluid.default_main_program()) - - server_io.save_model("ctr_serving_model_kv", "ctr_client_conf_kv", - feed_kv_dict, fetch_var_dict, - fluid.default_main_program()) - - -if __name__ == '__main__': - train() diff --git a/python/examples/criteo_ctr_with_cube/network_conf.py b/python/examples/criteo_ctr_with_cube/network_conf.py deleted file mode 100755 index 2975533a72ad21d6dd5896446fd06c1f9bdfe8b4..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/network_conf.py +++ /dev/null @@ -1,77 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -import paddle.fluid as fluid -import math - - -def dnn_model(dense_input, sparse_inputs, label, embedding_size, - sparse_feature_dim): - def embedding_layer(input): - emb = fluid.layers.embedding( - input=input, - is_sparse=True, - is_distributed=False, - size=[sparse_feature_dim, embedding_size], - param_attr=fluid.ParamAttr( - name="SparseFeatFactors", - initializer=fluid.initializer.Uniform())) - x = fluid.layers.sequence_pool(input=emb, pool_type='sum') - return emb, x - - def mlp_input_tensor(emb_sums, dense_tensor): - #if isinstance(dense_tensor, fluid.Variable): - # return fluid.layers.concat(emb_sums, axis=1) - #else: - return fluid.layers.concat(emb_sums + [dense_tensor], axis=1) - - def mlp(mlp_input): - fc1 = fluid.layers.fc(input=mlp_input, - size=400, - act='relu', - param_attr=fluid.ParamAttr( - initializer=fluid.initializer.Normal( - scale=1 / math.sqrt(mlp_input.shape[1])))) - fc2 = fluid.layers.fc(input=fc1, - size=400, - act='relu', - param_attr=fluid.ParamAttr( - initializer=fluid.initializer.Normal( - scale=1 / math.sqrt(fc1.shape[1])))) - fc3 = fluid.layers.fc(input=fc2, - size=400, - act='relu', - param_attr=fluid.ParamAttr( - initializer=fluid.initializer.Normal( - scale=1 / math.sqrt(fc2.shape[1])))) - pre = fluid.layers.fc(input=fc3, - size=2, - act='softmax', - param_attr=fluid.ParamAttr( - initializer=fluid.initializer.Normal( - scale=1 / math.sqrt(fc3.shape[1])))) - return pre - - emb_pair_sums = list(map(embedding_layer, sparse_inputs)) - emb_sums = [x[1] for x in emb_pair_sums] - infer_vars = [x[0] for x in emb_pair_sums] - mlp_in = mlp_input_tensor(emb_sums, dense_input) - predict = mlp(mlp_in) - cost = fluid.layers.cross_entropy(input=predict, label=label) - avg_cost = fluid.layers.reduce_sum(cost) - accuracy = fluid.layers.accuracy(input=predict, label=label) - auc_var, batch_auc_var, auc_states = \ - fluid.layers.auc(input=predict, label=label, num_thresholds=2 ** 12, slide_steps=20) - return predict, avg_cost, auc_var, batch_auc_var, infer_vars diff --git a/python/examples/criteo_ctr_with_cube/test_client.py b/python/examples/criteo_ctr_with_cube/test_client.py deleted file mode 100755 index 853b8fb5e793d7daeff4703f32c57cb57a9c279c..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/test_client.py +++ /dev/null @@ -1,59 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -from paddle_serving_client import Client -import sys -import os -import criteo as criteo -import time -from paddle_serving_client.metric import auc -import numpy as np - -py_version = sys.version_info[0] - -client = Client() -client.load_client_config(sys.argv[1]) -client.connect(["127.0.0.1:9292"]) - -batch = 1 -buf_size = 100 -dataset = criteo.CriteoDataset() -dataset.setup(1000001) -test_filelists = ["{}/part-0".format(sys.argv[2])] -reader = dataset.infer_reader(test_filelists, batch, buf_size) -label_list = [] -prob_list = [] -start = time.time() -for ei in range(10000): - if py_version == 2: - data = reader().next() - else: - data = reader().__next__() - feed_dict = {} - feed_dict['dense_input'] = np.array(data[0][0]).astype("float32").reshape( - 1, 13) - feed_dict['dense_input.lod'] = [0, 1] - for i in range(1, 27): - tmp_data = np.array(data[0][i]).astype(np.int64) - feed_dict["embedding_{}.tmp_0".format(i - 1)] = tmp_data.reshape( - (1, len(data[0][i]))) - feed_dict["embedding_{}.tmp_0.lod".format(i - 1)] = [0, 1] - fetch_map = client.predict(feed=feed_dict, fetch=["prob"], batch=True) - prob_list.append(fetch_map['prob'][0][1]) - label_list.append(data[0][-1][0]) - -print(auc(label_list, prob_list)) -end = time.time() -print(end - start) diff --git a/python/examples/criteo_ctr_with_cube/test_server.py b/python/examples/criteo_ctr_with_cube/test_server.py deleted file mode 100755 index 479c602910b5afa52b35a66d00316f54905c0741..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/test_server.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -import os -import sys -from paddle_serving_server import OpMaker -from paddle_serving_server import OpSeqMaker -from paddle_serving_server import Server - -op_maker = OpMaker() -read_op = op_maker.create('general_reader') -general_dist_kv_infer_op = op_maker.create('general_dist_kv_infer') -response_op = op_maker.create('general_response') - -op_seq_maker = OpSeqMaker() -op_seq_maker.add_op(read_op) -op_seq_maker.add_op(general_dist_kv_infer_op) -op_seq_maker.add_op(response_op) - -server = Server() -server.set_op_sequence(op_seq_maker.get_op_sequence()) -server.set_num_threads(4) -server.load_model_config(sys.argv[1]) -server.prepare_server( - workdir="work_dir1", - port=9292, - device="cpu", - cube_conf="./cube/conf/cube.conf") -server.run_server() diff --git a/python/examples/criteo_ctr_with_cube/test_server_gpu.py b/python/examples/criteo_ctr_with_cube/test_server_gpu.py deleted file mode 100755 index 33f74f91c13fca489db70a4d4171ae756355c787..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/test_server_gpu.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -import os -import sys -from paddle_serving_server_gpu import OpMaker -from paddle_serving_server_gpu import OpSeqMaker -from paddle_serving_server_gpu import Server - -op_maker = OpMaker() -read_op = op_maker.create('general_reader') -general_dist_kv_infer_op = op_maker.create('general_dist_kv_infer') -response_op = op_maker.create('general_response') - -op_seq_maker = OpSeqMaker() -op_seq_maker.add_op(read_op) -op_seq_maker.add_op(general_dist_kv_infer_op) -op_seq_maker.add_op(response_op) - -server = Server() -server.set_op_sequence(op_seq_maker.get_op_sequence()) -server.set_num_threads(4) -server.load_model_config(sys.argv[1]) -server.prepare_server( - workdir="work_dir1", - port=9292, - device="cpu", - cube_conf="./cube/conf/cube.conf") -server.run_server() diff --git a/python/examples/criteo_ctr_with_cube/test_server_quant.py b/python/examples/criteo_ctr_with_cube/test_server_quant.py deleted file mode 100755 index 38a3fe67da803d1c84337d64e3421d8295ac5767..0000000000000000000000000000000000000000 --- a/python/examples/criteo_ctr_with_cube/test_server_quant.py +++ /dev/null @@ -1,41 +0,0 @@ -# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# pylint: disable=doc-string-missing - -import os -import sys -from paddle_serving_server import OpMaker -from paddle_serving_server import OpSeqMaker -from paddle_serving_server import Server - -op_maker = OpMaker() -read_op = op_maker.create('general_reader') -general_dist_kv_infer_op = op_maker.create('general_dist_kv_quant_infer') -response_op = op_maker.create('general_response') - -op_seq_maker = OpSeqMaker() -op_seq_maker.add_op(read_op) -op_seq_maker.add_op(general_dist_kv_infer_op) -op_seq_maker.add_op(response_op) - -server = Server() -server.set_op_sequence(op_seq_maker.get_op_sequence()) -server.set_num_threads(4) -server.load_model_config(sys.argv[1]) -server.prepare_server( - workdir="work_dir1", - port=9292, - device="cpu", - cube_conf="./cube/conf/cube.conf") -server.run_server() diff --git a/python/pipeline/pipeline_server.py b/python/pipeline/pipeline_server.py index ed86adc1a1b23010051d897271390ab4b4f37d93..3f1157803e4e885db962a32837b09e8afbf14f96 100644 --- a/python/pipeline/pipeline_server.py +++ b/python/pipeline/pipeline_server.py @@ -333,7 +333,7 @@ class ServerYamlConfChecker(object): raise SystemExit("Failed to prepare_server: only one of yml_file" " or yml_dict can be selected as the parameter.") if yml_file is not None: - with open(yml_file) as f: + with open(yml_file, encoding='utf-8') as f: conf = yaml.load(f.read()) elif yml_dict is not None: conf = yml_dict