diff --git a/README.md b/README.md index f20012d0b9c47965e50be0bde6158ac8912419d7..b31f72af51b005d260bc07bbb28a1408e8ee91a6 100644 --- a/README.md +++ b/README.md @@ -65,6 +65,7 @@ python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --po | `port` | int | `9292` | Exposed port of current service to users| | `name` | str | `""` | Service name, can be used to generate HTTP request url | | `model` | str | `""` | Path of paddle model directory to be served | +| `mem_optim` | bool | `False` | Enable memory optimization | Here, we use `curl` to send a HTTP POST request to the service we just started. Users can use any python library to send HTTP POST as well, e.g, [requests](https://requests.readthedocs.io/en/master/). @@ -222,7 +223,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv | Key | Value | | :----------------- | :----------------------------------------------------------- | | Model Name | DNN-CTR | -| URL | None(Get model by [local_train.py](./python/examples/criteo_ctr/local_train.py)) | +| URL | https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz | | Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr | | Description | Get click probability from a feature vector of item | @@ -231,7 +232,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv | Key | Value | | :----------------- | :----------------------------------------------------------- | | Model Name | DNN-CTR(with cube) | -| URL | None(Get model by [local_train.py](python/examples/criteo_ctr_with_cube/local_train.py)) | +| URL | https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz | | Client/Server Code | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr_with_cube | | Description | Get click probability from a feature vector of item | @@ -249,7 +250,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv - [Compile from source code](doc/COMPILE.md) ### About Efficiency -- [How to profile Paddle Serving latency?](https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/util) +- [How to profile Paddle Serving latency?](python/examples/util) - [CPU Benchmarks(Chinese)](doc/BENCHMARKING.md) - [GPU Benchmarks(Chinese)](doc/GPU_BENCHMARKING.md) diff --git a/README_CN.md b/README_CN.md index f5ca91ef597bad138b110eb9af41f40c19b87adc..80c6041205ec91482f7d30157efa5e4e66a7c021 100644 --- a/README_CN.md +++ b/README_CN.md @@ -54,6 +54,41 @@ tar -xzf uci_housing.tar.gz Paddle Serving 为用户提供了基于 HTTP 和 RPC 的服务 +

HTTP服务

+ +Paddle Serving提供了一个名为`paddle_serving_server.serve`的内置python模块，可以使用单行命令启动RPC服务或HTTP服务。如果我们指定参数`--name uci`，则意味着我们将拥有一个HTTP服务，其URL为$IP:$PORT/uci/prediction`。 + +``` shell +python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 --name uci +``` +
+ +| Argument | Type | Default | Description | +|--------------|------|-----------|--------------------------------| +| `thread` | int | `4` | Concurrency of current service | +| `port` | int | `9292` | Exposed port of current service to users| +| `name` | str | `""` | Service name, can be used to generate HTTP request url | +| `model` | str | `""` | Path of paddle model directory to be served | +| `mem_optim` | bool | `False` | Enable memory optimization | + +我们使用 `curl` 命令来发送HTTP POST请求给刚刚启动的服务。用户也可以调用python库来发送HTTP POST请求，请参考英文文档 [requests](https://requests.readthedocs.io/en/master/)。 +
+ +``` shell +curl -H "Content-Type:application/json" -X POST -d '{"x": [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332], "fetch":["price"]}' http://127.0.0.1:9292/uci/prediction +``` + +

RPC服务

+ +用户还可以使用`paddle_serving_server.serve`启动RPC服务。 尽管用户需要基于Paddle Serving的python客户端API进行一些开发，但是RPC服务通常比HTTP服务更快。需要指出的是这里我们没有指定`--name`。 + +``` shell +python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292 +``` + +Paddle Serving 为用户提供了基于 HTTP 和 RPC 的服务 + +

HTTP服务

Paddle Serving提供了一个名为`paddle_serving_server.serve`的内置python模块，可以使用单行命令启动RPC服务或HTTP服务。如果我们指定参数`--name uci`，则意味着我们将拥有一个HTTP服务，其URL为$IP:$PORT/uci/prediction`。 @@ -227,7 +262,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv | Key | Value | | :----------------- | :----------------------------------------------------------- | | 模型名 | DNN-CTR | -| 下载链接 | None(Get model by [local_train.py](./python/examples/criteo_ctr/local_train.py)) | +| 下载链接 | https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz | | 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr | | 介绍 | 从项目的特征向量中获得点击概率 | @@ -236,7 +271,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv | Key | Value | | :----------------- | :----------------------------------------------------------- | | 模型名 | DNN-CTR(with cube) | -| 下载链接 | None(Get model by [local_train.py](python/examples/criteo_ctr_with_cube/local_train.py)) | +| 下载链接 | https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz | | 客户端/服务端代码 | https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/criteo_ctr_with_cube | | 介绍 | 从项目的特征向量中获得点击概率 | @@ -255,7 +290,7 @@ curl -H "Content-Type:application/json" -X POST -d '{"url": "https://paddle-serv - [如何编译PaddleServing?](doc/COMPILE_CN.md) ### 关于Paddle Serving性能 -- [如何测试Paddle Serving性能？](https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/util/) +- [如何测试Paddle Serving性能？](python/examples/util/) - [CPU版Benchmarks](doc/BENCHMARKING.md) - [GPU版Benchmarks](doc/GPU_BENCHMARKING.md) diff --git a/cmake/generic.cmake b/cmake/generic.cmake index 03066aca2b85e9c5473e1ec0dae648fdd86e41ad..861889266b0132b8812d2d958dd6675dc631fd33 100644 --- a/cmake/generic.cmake +++ b/cmake/generic.cmake @@ -832,6 +832,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc") list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h") + set(PDCODEGEN "${CMAKE_BINARY_DIR}/core/pdcodegen/pdcodegen") if (${FOR_SERVING_SIDE}) add_custom_command( OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc" @@ -842,7 +843,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) --plugin=protoc-gen-pdcodegen=${CMAKE_BINARY_DIR}/core/pdcodegen/pdcodegen --proto_path=${CMAKE_SOURCE_DIR}/core/predictor/proto ${_protobuf_include_path} ${ABS_FIL} - DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} + DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} ${PDCODEGEN} COMMENT "Running Paddle-serving C++ protocol buffer compiler on ${FIL}" VERBATIM) else() @@ -854,7 +855,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) --pdcodegen_out=${CMAKE_CURRENT_BINARY_DIR} --plugin=protoc-gen-pdcodegen=${CMAKE_BINARY_DIR}/pdcodegen/pdcodegen ${_protobuf_include_path} ${ABS_FIL} - DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} + DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} ${PDCODEGEN} COMMENT "Running Paddle-serving C++ protocol buffer compiler on ${FIL}" VERBATIM) endif() diff --git a/core/sdk-cpp/proto/CMakeLists.txt b/core/sdk-cpp/proto/CMakeLists.txt index 5bc378398f341025d48b1d39d22785305aba2dea..e0fb6aad3dba9491cde76017550f248f1e573e90 100644 --- a/core/sdk-cpp/proto/CMakeLists.txt +++ b/core/sdk-cpp/proto/CMakeLists.txt @@ -37,6 +37,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) list(APPEND ${SRCS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc") list(APPEND ${HDRS} "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.h") + set(PDCODEGEN "${CMAKE_BINARY_DIR}/core/pdcodegen/pdcodegen") if (${FOR_SERVING_SIDE}) add_custom_command( OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/${FIL_WE}.pb.cc" @@ -47,7 +48,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) --plugin=protoc-gen-pdcodegen=${CMAKE_BINARY_DIR}/core/pdcodegen/pdcodegen --proto_path=${CMAKE_SOURCE_DIR}/core/predictor/proto ${_protobuf_include_path} ${ABS_FIL} - DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} + DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} ${PDCODEGEN} COMMENT "Running Paddle-serving C++ protocol buffer compiler on ${FIL}" VERBATIM) else() @@ -59,7 +60,7 @@ function(PROTOBUF_GENERATE_SERVING_CPP FOR_SERVING_SIDE SRCS HDRS ) --pdcodegen_out=${CMAKE_CURRENT_BINARY_DIR} --plugin=protoc-gen-pdcodegen=${CMAKE_BINARY_DIR}/core/pdcodegen/pdcodegen ${_protobuf_include_path} ${ABS_FIL} - DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} + DEPENDS ${ABS_FIL} ${Protobuf_PROTOC_EXECUTABLE} ${PDCODEGEN} COMMENT "Running Paddle-serving C++ protocol buffer compiler on ${FIL}" VERBATIM) endif() diff --git a/core/sdk-cpp/src/config_manager.cpp b/core/sdk-cpp/src/config_manager.cpp index c422f0b52eba7d3a34e663f4198b9914a7722704..e3126855e082feaf9c6d237692c214fa8f66577b 100644 --- a/core/sdk-cpp/src/config_manager.cpp +++ b/core/sdk-cpp/src/config_manager.cpp @@ -31,6 +31,8 @@ int EndpointConfigManager::create(const std::string& sdk_desc_str) { LOG(ERROR) << "Failed reload endpoint config"; return -1; } + + return 0; } int EndpointConfigManager::create(const char* path, const char* file) { diff --git a/doc/SERVER_DAG.md b/doc/SERVER_DAG.md index fdfcec948e3224ba53c4ab09d0551b3df205e8aa..5a5c851efacc28e5419d262ca671c83ec61e2015 100644 --- a/doc/SERVER_DAG.md +++ b/doc/SERVER_DAG.md @@ -14,7 +14,7 @@ Deep neural nets often have some preprocessing steps on input data, and postproc ## How to define Node -PaddleServing has some predefined Computation Node in the framework. A very commonly used Computation Graph is the simple reader-inference-response mode that can cover most of the single model inference scenarios. A example graph and the corresponding DAG defination code is as follows. +PaddleServing has some predefined Computation Node in the framework. A very commonly used Computation Graph is the simple reader-inference-response mode that can cover most of the single model inference scenarios. A example graph and the corresponding DAG definition code is as follows.
diff --git a/doc/TRAIN_TO_SERVICE.md b/doc/TRAIN_TO_SERVICE.md index 4219e66948a9bc3b0ae43e5cda61aad8ae35b3a0..c0d7d2ea3794a07e39407bb0b53a822cfedd173e 100644 --- a/doc/TRAIN_TO_SERVICE.md +++ b/doc/TRAIN_TO_SERVICE.md @@ -5,9 +5,9 @@ Paddle Serving is Paddle's high-performance online inference service framework, which can flexibly support the deployment of most models. In this article, the IMDB review sentiment analysis task is used as an example to show the entire process from model training to deployment of inference service through 9 steps. ## Step1：Prepare for Running Environment -Paddle Serving can be deployed on Linux environments such as Centos and Ubuntu. On other systems or in environments where you do not want to install the serving module, you can still access the server-side prediction service through the http service. +Paddle Serving can be deployed on Linux environments.Currently the server supports deployment on Centos7. [Docker deployment is recommended](RUN_IN_DOCKER.md). The rpc client supports deploymen on Centos7 and Ubuntu 18.On other systems or in environments where you do not want to install the serving module, you can still access the server-side prediction service through the http service. -You can choose to install the cpu or gpu version of the server module according to the requirements and machine environment, and install the client module on the client machine. When you want to access the server with http +You can choose to install the cpu or gpu version of the server module according to the requirements and machine environment, and install the client module on the client machine. When you want to access the server with http, there is not need to install client module. ```shell pip install paddle_serving_server #cpu version server side @@ -228,7 +228,7 @@ if __name__ == "__main__": -! [Training process](./ imdb_loss.png) As can be seen from the above figure, the loss of the model starts to converge after the 65th round. We save the model and configuration file after the 65th round of training is completed. The saved files are divided into imdb_cnn_client_conf and imdb_cnn_model folders. The former contains client-side configuration files, and the latter contains server-side configuration files and saved model files. +![Training process](./imdb_loss.png) As can be seen from the above figure, the loss of the model starts to converge after the 65th round. We save the model and configuration file after the 65th round of training is completed. The saved files are divided into imdb_cnn_client_conf and imdb_cnn_model folders. The former contains client-side configuration files, and the latter contains server-side configuration files and saved model files. The parameter list of the save_model function is as follows: | Parameter | Meaning | @@ -243,10 +243,10 @@ The parameter list of the save_model function is as follows: The Paddle Serving framework supports two types of prediction service methods. One is to communicate through RPC and the other is to communicate through HTTP. The deployment and use of RPC prediction service will be introduced first. The deployment and use of HTTP prediction service will be introduced at Step 8. . -`` `shell +```shell python -m paddle_serving_server.serve --model imdb_cnn_model / --port 9292 #cpu prediction service python -m paddle_serving_server_gpu.serve --model imdb_cnn_model / --port 9292 --gpu_ids 0 #gpu prediction service -`` ` +``` The parameter --model in the command specifies the server-side model and configuration file directory previously saved, --port specifies the port of the prediction service. When deploying the gpu prediction service using the gpu version, you can use --gpu_ids to specify the gpu used. @@ -287,13 +287,13 @@ The script receives data from standard input and prints out the probability that The client implemented in the previous step runs the prediction service as an example. The usage method is as follows: -`` `shell +```shell cat test_data/part-0 | python test_client.py imdb_lstm_client_conf / serving_client_conf.prototxt imdb.vocab -`` ` +``` Using 2084 samples in the test_data/part-0 file for test testing, the model prediction accuracy is 88.19%. -** Note **: The effect of each model training may be slightly different, and the accuracy of predictions using the trained model will be close to the examples but may not be exactly the same. +**Note**: The effect of each model training may be slightly different, and the accuracy of predictions using the trained model will be close to the examples but may not be exactly the same. ## Step8: Deploy HTTP Prediction Service @@ -349,13 +349,13 @@ In the above command, the first parameter is the saved server-side model and con ## Step9: Call the prediction service with plaintext data After starting the HTTP prediction service, you can make prediction with a single command: -`` ` +``` curl -H "Content-Type: application / json" -X POST -d '{"words": "i am very sad | 0", "fetch": ["prediction"]}' http://127.0.0.1:9292/imdb/prediction -`` ` +``` When the inference process is normal, the prediction probability is returned, as shown below. -`` ` +``` {"prediction": [0.5592559576034546,0.44074398279190063]} -`` ` +``` -** Note **: The effect of each model training may be slightly different, and the inferred probability value using the trained model may not be consistent with the example. +**Note**: The effect of each model training may be slightly different, and the inferred probability value using the trained model may not be consistent with the example. diff --git a/doc/TRAIN_TO_SERVICE_CN.md b/doc/TRAIN_TO_SERVICE_CN.md index 8349723fb3a749efcbcc5887ff5f7ba1ede7ad65..14a3ee9ce12a19f0274357e525efb27f758b3c97 100644 --- a/doc/TRAIN_TO_SERVICE_CN.md +++ b/doc/TRAIN_TO_SERVICE_CN.md @@ -6,9 +6,11 @@ Paddle Serving是Paddle的高性能在线预测服务框架，可以灵活支持 ## Step1：准备环境 -Paddle Serving可以部署在Centos和Ubuntu等Linux环境上，在其他系统上或者不希望安装serving模块的环境中仍然可以通过http服务来访问server端的预测服务。 -可以根据需求和机器环境来选择安装cpu或gpu版本的server模块，在client端机器上安装client模块。当希望同http来访问server端 +Paddle Serving可以部署在Linux环境上，目前server端支持在Centos7上部署，推荐使用[Docker部署](RUN_IN_DOCKER_CN.md)。rpc client端可以在Centos7和Ubuntu18上部署，在其他系统上或者不希望安装serving模块的环境中仍然可以通过http服务来访问server端的预测服务。 + +可以根据需求和机器环境来选择安装cpu或gpu版本的server模块，在client端机器上安装client模块。使用http请求的方式来访问server时，client端机器不需要安装client模块。 + ```shell pip install paddle_serving_server #cpu版本server端 diff --git a/python/examples/criteo_ctr/README.md b/python/examples/criteo_ctr/README.md index e59947ba377160fe0dc1d1105f35c467e50d32ad..4780fb667dfe2a0bc4bc4497dc24a495b59aa3ac 100644 --- a/python/examples/criteo_ctr/README.md +++ b/python/examples/criteo_ctr/README.md @@ -1,26 +1,31 @@ -## CTR预测服务 +## CTR Prediction Service -### 获取样例数据 +([简体中文](./README_CN.md)|English) + +### download criteo dataset ``` sh get_data.sh ``` -### 保存模型和配置文件 +### download inference model ``` -python local_train.py +wget https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz +tar xf criteo_ctr_demo_model.tar.gz +mv models/ctr_client_conf . +mv models/ctr_serving_model . ``` -执行脚本后会在当前目录生成serving_server_model和serving_client_config文件夹。 +the directories like serving_server_model and serving_client_config will appear. -### 启动RPC预测服务 +### Start RPC Inference Service ``` -python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #启动CPU预测服务 -python -m paddle_serving_server_gpu.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #在GPU 0上启动预测服务 +python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #CPU RPC Service +python -m paddle_serving_server_gpu.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #RPC Service on GPU 0 ``` -### 执行预测 +### RPC Infer ``` python test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/ ``` -预测完毕会输出预测过程的耗时。 +the latency will display in the end. diff --git a/python/examples/criteo_ctr/README_CN.md b/python/examples/criteo_ctr/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..bee946aac8eca4293811057baff061bceb9508af --- /dev/null +++ b/python/examples/criteo_ctr/README_CN.md @@ -0,0 +1,31 @@ +## CTR预测服务 + +(简体中文|[English](./README.md)) + +### 获取样例数据 +``` +sh get_data.sh +``` + +### 下载模型 +``` +wget https://paddle-serving.bj.bcebos.com/criteo_ctr_example/criteo_ctr_demo_model.tar.gz +tar xf criteo_ctr_demo_model.tar.gz +mv models/ctr_client_conf . +mv models/ctr_serving_model . +``` +会在当前目录出现serving_server_model和serving_client_config文件夹。 + +### 启动RPC预测服务 + +``` +python -m paddle_serving_server.serve --model ctr_serving_model/ --port 9292 #启动CPU预测服务 +python -m paddle_serving_server_gpu.serve --model ctr_serving_model/ --port 9292 --gpu_ids 0 #在GPU 0上启动预测服务 +``` + +### 执行预测 + +``` +python test_client.py ctr_client_conf/serving_client_conf.prototxt raw_data/ +``` +预测完毕会输出预测过程的耗时。 diff --git a/python/examples/criteo_ctr_with_cube/README.md b/python/examples/criteo_ctr_with_cube/README.md index eed612f4043ff7fdb41538b2a425d98f0d045718..25f171f5b07a6c58de68809ae6092c85f92b8116 100755 --- a/python/examples/criteo_ctr_with_cube/README.md +++ b/python/examples/criteo_ctr_with_cube/README.md @@ -2,21 +2,35 @@ ([简体中文](./README_CN.md)|English) +### Compile Source Code +in the root directory of this git project +``` +mkdir build_server +cd build_server +cmake -DPYTHON_INCLUDE_DIR=$PYTHONROOT/include/python2.7/ -DPYTHON_LIBRARIES=$PYTHONROOT/lib64/libpython2.7.so -DPYTHON_EXECUTABLE=$PYTHONROOT/bin/python -DSERVER=ON .. +make -j10 +make install -j10 +``` + ### Get Sample Dataset +go to directory `python/examples/criteo_ctr_with_cube` ``` sh get_data.sh ``` -### Train and Save Model +### Download Model and Sparse Parameter Sequence Files ``` -python local_train.py +wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz +tar xf ctr_cube_unittest.tar.gz +mv models/ctr_client_conf ./ +mv models/ctr_serving_model_kv ./ +mv models/data ./cube/ ``` -the trained model will be in ./ctr_server_model and ./ctr_client_config, and ctr_server_model_kv, ctr_client_conf_kv。 +the model will be in ./ctr_server_model_kv and ./ctr_client_config. ### Start Sparse Parameter Indexing Service ``` -cp ../../../build_server/core/predictor/seq_generator seq_generator cp ../../../build_server/output/bin/cube* ./cube/ sh cube_prepare.sh & ``` diff --git a/python/examples/criteo_ctr_with_cube/README_CN.md b/python/examples/criteo_ctr_with_cube/README_CN.md index 868e8bce5a624904d532bf956fd5868abc0a1c52..47279cc8c2dd781324fe4c73d98f58cbd69319c9 100644 --- a/python/examples/criteo_ctr_with_cube/README_CN.md +++ b/python/examples/criteo_ctr_with_cube/README_CN.md @@ -1,20 +1,34 @@ ## 带稀疏参数索引服务的CTR预测服务 (简体中文|[English](./README.md)) +### 编译源代码 +在本项目的根目录下，执行 +``` +mkdir build_server +cd build_server +cmake -DPYTHON_INCLUDE_DIR=$PYTHONROOT/include/python2.7/ -DPYTHON_LIBRARIES=$PYTHONROOT/lib64/libpython2.7.so -DPYTHON_EXECUTABLE=$PYTHONROOT/bin/python -DSERVER=ON .. +make -j10 +make install -j10 +``` + ### 获取样例数据 +进入目录 `python/examples/criteo_ctr_with_cube` ``` sh get_data.sh ``` -### 保存模型和配置文件 +### 下载模型和稀疏参数序列文件 ``` -python local_train.py +wget https://paddle-serving.bj.bcebos.com/unittest/ctr_cube_unittest.tar.gz +tar xf ctr_cube_unittest.tar.gz +mv models/ctr_client_conf ./ +mv models/ctr_serving_model_kv ./ +mv models/data ./cube/ ``` -执行脚本后会在当前目录生成ctr_server_model和ctr_client_config文件夹,以及ctr_server_model_kv, ctr_client_conf_kv。 +执行脚本后会在当前目录有ctr_server_model_kv和ctr_client_config文件夹。 ### 启动稀疏参数索引服务 ``` -cp ../../../build_server/core/predictor/seq_generator seq_generator cp ../../../build_server/output/bin/cube* ./cube/ sh cube_prepare.sh & ``` diff --git a/python/examples/lac/README.md b/python/examples/lac/README.md new file mode 100644 index 0000000000000000000000000000000000000000..a0553b24ab377ed7d274583ed84827f2f1a985af --- /dev/null +++ b/python/examples/lac/README.md @@ -0,0 +1,32 @@ +## Chinese Word Segmentation + +([简体中文](./README_CN.md)|English) + +### Get model files and sample data +``` +sh get_data.sh +``` + +the package downloaded contains lac model config along with lac dictionary. + +#### Start RPC inference service + +``` +python -m paddle_serving_server.serve --model jieba_server_model/ --port 9292 +``` +### RPC Infer +``` +echo "我爱北京天安门" | python lac_client.py jieba_client_conf/serving_client_conf.prototxt lac_dict/ +``` + +it will get the segmentation result + +### Start HTTP inference service +``` +python lac_web_service.py jieba_server_model/ lac_workdir 9292 +``` +### HTTP Infer + +``` +curl -H "Content-Type:application/json" -X POST -d '{"words": "我爱北京天安门", "fetch":["word_seg"]}' http://127.0.0.1:9292/lac/prediction +``` diff --git a/python/examples/lac/README_CN.md b/python/examples/lac/README_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..98f2d36497dbf5dea8e34de355ae96a7f529349a --- /dev/null +++ b/python/examples/lac/README_CN.md @@ -0,0 +1,32 @@ +## 中文分词模型 + +(简体中文|[English](./README.md)) + +### 获取模型和字典文件 +``` +sh get_data.sh +``` + +下载包里包含了lac模型和lac模型预测需要的字典文件 + +#### 开启RPC预测服务 + +``` +python -m paddle_serving_server.serve --model jieba_server_model/ --port 9292 +``` +### 执行RPC预测 +``` +echo "我爱北京天安门" | python lac_client.py jieba_client_conf/serving_client_conf.prototxt lac_dict/ +``` + +我们就能得到分词结果 + +### 开启HTTP预测服务 +``` +python lac_web_service.py jieba_server_model/ lac_workdir 9292 +``` +### 执行HTTP预测 + +``` +curl -H "Content-Type:application/json" -X POST -d '{"words": "我爱北京天安门", "fetch":["word_seg"]}' http://127.0.0.1:9292/lac/prediction +``` diff --git a/python/examples/lac/get_data.sh b/python/examples/lac/get_data.sh index 6b72850d35b7a7b5e43b34d31c7a903e05f07440..29e6a6b2b3e995f78c37e15baf2f9a3b627ca9ef 100644 --- a/python/examples/lac/get_data.sh +++ b/python/examples/lac/get_data.sh @@ -1,2 +1,2 @@ -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/lac/lac_model.tar.gz -tar -zxvf lac_model.tar.gz +wget --no-check-certificate https://paddle-serving.bj.bcebos.com/lac/lac_model_jieba_web.tar.gz +tar -zxvf lac_model_jieba_web.tar.gz diff --git a/python/examples/lac/lac_client.py b/python/examples/lac/lac_client.py index f2a8e858ed72ac4043a2bb3162a39a2aff233043..9c485a923e4d42b72af41f7b9ad45c5702ca93a1 100644 --- a/python/examples/lac/lac_client.py +++ b/python/examples/lac/lac_client.py @@ -22,7 +22,7 @@ import io client = Client() client.load_client_config(sys.argv[1]) -client.connect(["127.0.0.1:9280"]) +client.connect(["127.0.0.1:9292"]) reader = LACReader(sys.argv[2]) for line in sys.stdin: diff --git a/python/examples/lac/lac_reader.py b/python/examples/lac/lac_reader.py index 087ec8bb9e1a44afa2ba5a1cc9931e350aa76fb7..0c44177c2d56e5de94a18ce3514d0439a33361c5 100644 --- a/python/examples/lac/lac_reader.py +++ b/python/examples/lac/lac_reader.py @@ -99,3 +99,26 @@ class LACReader(object): words = sent.strip() word_ids = self.word_to_ids(words) return word_ids + + def parse_result(self, words, crf_decode): + tags = [self.id2label_dict[str(x)] for x in crf_decode] + + sent_out = [] + tags_out = [] + partial_word = "" + for ind, tag in enumerate(tags): + if partial_word == "": + partial_word = words[ind] + tags_out.append(tag.split('-')[0]) + continue + if tag.endswith("-B") or (tag == "O" and tag[ind - 1] != "O"): + sent_out.append(partial_word) + tags_out.append(tag.split('-')[0]) + partial_word = words[ind] + continue + partial_word += words[ind] + + if len(sent_out) < len(tags_out): + sent_out.append(partial_word) + + return sent_out diff --git a/python/examples/lac/lac_web_service.py b/python/examples/lac/lac_web_service.py index 4a58c6a43caea4045220546488226da121bfdc17..186d8badf8806606998466e3d1bb4047bf51b5d8 100644 --- a/python/examples/lac/lac_web_service.py +++ b/python/examples/lac/lac_web_service.py @@ -25,8 +25,13 @@ class LACService(WebService): if "words" not in feed: raise ("feed data error!") feed_data = self.reader.process(feed["words"]) + fetch = ["crf_decode"] return {"words": feed_data}, fetch + def postprocess(self, feed={}, fetch=[], fetch_map={}): + segs = self.reader.parse_result(feed["words"], fetch_map["crf_decode"]) + return {"word_seg": "|".join(segs)} + lac_service = LACService(name="lac") lac_service.load_model_config(sys.argv[1])