diff --git a/doc/ENCRYPTION.md b/doc/ENCRYPTION.md new file mode 100644 index 0000000000000000000000000000000000000000..1e6a53aa386bf672d5f87647cb1682531ea3d62c --- /dev/null +++ b/doc/ENCRYPTION.md @@ -0,0 +1,52 @@ +# MOEDL ENCRYPTION INFERENCE + +([简体中文](ENCRYPTION_CN.md)|English) + +Paddle Serving provides model encryption inference, This document shows the details. + +## Principle + +We use symmetric encryption algorithm to encrypt the model. Symmetric encryption algorithm uses the same key for encryption and decryption, it has small amount of calculation, fast speed, is the most commonly used encryption method. + +### Got an Encrypted Model + +Normal model and parameters can be understood as a string, by using the encryption algorithm (parameter is your key) on them, the normal model and parameters become an encrypted one. + +We provide a simple demo to encrypt the model. See the [python/examples/encryption/encrypt.py](../python/examples/encryption/encrypt.py)。 + + +### Start Encryption Service + +Suppose you already have an encrypted model（in the `encrypt_server/`）,you can start the encryption model service by adding an additional command line parameter `--use_encryption_model` + +CPU Service +``` +python -m paddle_serving_server.serve --model encrypt_server/ --port 9300 --use_encryption_model +``` +GPU Service +``` +python -m paddle_serving_server_gpu.serve --model encrypt_server/ --port 9300 --use_encryption_model --gpu_ids 0 +``` + +At this point, the server does not really start, but waits for the key。 + +### Client Encryption Inference + +First of all, you got have the key which is used in the process of model encryption. + +Then you can configure your client with the key, when you connect the server, this key will send to the server and the server will keep it. + +Once the server gets the key, it uses the key to parse the model and starts the model prediction service. + + +### Example of Model Encryption Inference +Example of model encryption inference, See the [`/python/examples/encryption/`](../python/examples/encryption/)。 + + +### Other Details +Interface of encryption method in paddlepaddle official website: + +[Python encryption method](https://github.com/HexToString/Serving/blob/develop/python/paddle_serving_app/local_predict.py) + +[C++ encryption method](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/inference/python_infer_cn.html#analysispre) + diff --git a/doc/ENCRYPTION_CN.md b/doc/ENCRYPTION_CN.md new file mode 100644 index 0000000000000000000000000000000000000000..5ca304d00d198ba2c6df1c7cfbff7315ba46fe15 --- /dev/null +++ b/doc/ENCRYPTION_CN.md @@ -0,0 +1,52 @@ +# 加密模型预测 + +(简体中文|[English](ENCRYPTION.md)) + +Padle Serving提供了模型加密预测功能，本文档显示了详细信息。 + +## 原理 + +采用对称加密算法对模型进行加密。对称加密算法采用同一密钥进行加解密，它计算量小，速度快，是最常用的加密方法。 + +### 获得加密模型 + +普通的模型和参数可以理解为一个字符串，通过对其使用加密算法（参数是您的密钥），普通模型和参数就变成了一个加密的模型和参数。 + +我们提供了一个简单的演示来加密模型。请参阅[`python/examples/encryption/encrypt.py`](../python/examples/encryption/encrypt.py)。 + + +### 启动加密服务 + +假设您已经有一个已经加密的模型（在`encrypt_server/`路径下）,您可以通过添加一个额外的命令行参数 `--use_encryption_model`来启动加密模型服务。 + +CPU Service +``` +python -m paddle_serving_server.serve --model encrypt_server/ --port 9300 --use_encryption_model +``` +GPU Service +``` +python -m paddle_serving_server_gpu.serve --model encrypt_server/ --port 9300 --use_encryption_model --gpu_ids 0 +``` + +此时，服务器不会真正启动，而是等待密钥。 + +### Client Encryption Inference + +首先，您必须拥有模型加密过程中使用的密钥。 + +然后你可以用这个密钥配置你的客户端，当你连接服务器时，这个密钥会发送到服务器，服务器会保留它。 + +一旦服务器获得密钥，它就使用该密钥解析模型并启动模型预测服务。 + + +### 模型加密推理示例 +模型加密推理示例, 请参见[`/python/examples/encryption/`](../python/examples/encryption/)。 + + +### 其他详细信息 +飞桨官方网站加密方法接口 + +[Python加密方法接口](https://github.com/HexToString/Serving/blob/develop/python/paddle_serving_app/local_predict.py) + +[C++加密方法接口](https://www.paddlepaddle.org.cn/documentation/docs/zh/advanced_guide/inference_deployment/inference/python_infer_cn.html#analysispre) + diff --git a/doc/GRPC_IMPL_CN.md b/doc/GRPC_IMPL_CN.md index 9e7ecd268fe0900c1085479c1f96fa083629758c..7cfa9d86f7c92d0f33f2984c116993544159f7e8 100644 --- a/doc/GRPC_IMPL_CN.md +++ b/doc/GRPC_IMPL_CN.md @@ -24,13 +24,12 @@ #### 1.1 服务端对比 -* gRPC Server 端 `load_model_config` 函数添加 `client_config_path` 参数： +* 由于gRPC Server 端实际包含了brpc-Client端的，因此brpc-Client的初始化过程是在gRPC Server 端实现的，所以gRPC Server 端 `load_model_config` 函数添加 `client_config_path` 参数，用于指定brpc-Client初始化过程中的传输数据格式配置文件路径（`client_config_path` 参数未指定时默认为None,此时`client_config_path` 在`load_model_config` 函数中被默认为 `/serving_server_conf.prototxt`，此时brpc-Client与brpc-Server的传输数据格式配置文件相同） ``` def load_model_config(self, server_config_paths, client_config_path=None) ``` 在一些例子中 bRPC Server 端与 bRPC Client 端的配置文件可能不同（如 在cube local 中，Client 端的数据先交给 cube，经过 cube 处理后再交给预测库），此时 gRPC Server 端需要手动设置 gRPC Client 端的配置`client_config_path`。 - **`client_config_path` 默认为 `/serving_server_conf.prototxt`。** #### 1.2 客服端对比 @@ -47,13 +46,15 @@ * gRPC Client 端 `predict` 函数添加 `asyn` 和 `is_python` 参数： ``` - def predict(self, feed, fetch, need_variant_tag=False, asyn=False, is_python=True) + def predict(self, feed, fetch, batch=True, need_variant_tag=False, asyn=False, is_python=True,log_id=0) ``` 1. `asyn` 为异步调用选项。当 `asyn=True` 时为异步调用，返回 `MultiLangPredictFuture` 对象，通过 `MultiLangPredictFuture.result()` 阻塞获取预测值；当 `asyn=Fasle` 为同步调用。 2. `is_python` 为 proto 格式选项。当 `is_python=True` 时，基于 numpy bytes 格式进行数据传输，目前只适用于 Python；当 `is_python=False` 时，以普通数据格式传输，更加通用。使用 numpy bytes 格式传输耗时比普通数据格式小很多（详见 [#654](https://github.com/PaddlePaddle/Serving/pull/654)）。 +3. `batch`为数据是否需要进行增维处理的选项。当`batch=True`时，feed数据不需要额外的处理，维持原有维度；当`batch=False`时,会对数据进行增维度处理。例如：feed.shape原始为[2,2]，当`batch=False`时,会将feed.reshape为[1,2,2]。 + #### 1.3 其他 * 异常处理：当 gRPC Server 端的 bRPC Client 预测失败（返回 `None`）时，gRPC Client 端同样返回None。其他 gRPC 异常会在 Client 内部捕获，并在返回的 fetch_map 中添加一个 "status_code" 字段来区分是否预测正常（参考 timeout 样例）。 @@ -74,7 +75,7 @@ ## 2.示例：线性回归预测服务 -以下是采用gRPC实现的关于线性回归预测的一个示例，具体代码详见此[链接](https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/grpc_impl_example/fit_a_line) +以下是采用gRPC实现的关于线性回归预测的一个示例，具体代码详见此[链接](../python/examples/grpc_impl_example/fit_a_line) #### 获取数据 ```shell @@ -134,4 +135,4 @@ python test_list_input_client.py ## 3.更多示例 -详见[`python/examples/grpc_impl_example`](https://github.com/PaddlePaddle/Serving/tree/develop/python/examples/grpc_impl_example)下的示例文件。 +详见[`python/examples/grpc_impl_example`](../python/examples/grpc_impl_example)下的示例文件。 diff --git a/doc/JAVA_SDK.md b/doc/JAVA_SDK.md index 4880e74bfee123b432b6b583a239d2d2ccbb45ac..01da7156a6d1a803bd06171664fe9e2c4e977d83 100644 --- a/doc/JAVA_SDK.md +++ b/doc/JAVA_SDK.md @@ -20,6 +20,7 @@ The following table shows compatibilities between Paddle Serving Server and Java | :---------------------------: | :--------------: | | 0.3.2 | 0.0.1 | +1. Directly use the provided Java SDK as the client for prediction ### Install Java SDK You can download jar and install it to the local Maven repository: @@ -29,14 +30,6 @@ wget https://paddle-serving.bj.bcebos.com/jar/paddle-serving-sdk-java-0.0.1.jar mvn install:install-file -Dfile=$PWD/paddle-serving-sdk-java-0.0.1.jar -DgroupId=io.paddle.serving.client -DartifactId=paddle-serving-sdk-java -Dversion=0.0.1 -Dpackaging=jar ``` -Or compile from the source code and install it to the local Maven repository: - -```shell -cd Serving/java -mvn compile -mvn install -``` - ### Maven configure ```text @@ -47,63 +40,8 @@ mvn install ``` +2. Use it after compiling from the source code. See the [document](../java/README.md). -## Example - -Here we will show how to use Java SDK for Boston house price prediction. Please refer to [examples](../java/examples) folder for more examples. - -### Get model - -```shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz -``` +3. examples for using the java client, see the See the [document](../java/README.md). -### Start Python Server - -```shell -python -m paddle_serving_server.serve --model uci_housing_model --port 9393 --use_multilang -``` - -#### Client side code example - -```java -import io.paddle.serving.client.*; -import org.nd4j.linalg.api.ndarray.INDArray; -import org.nd4j.linalg.factory.Nd4j; -import java.util.*; - -public class PaddleServingClientExample { - public static void main( String[] args ) { - float[] data = {0.0137f, -0.1136f, 0.2553f, -0.0692f, - 0.0582f, -0.0727f, -0.1583f, -0.0584f, - 0.6283f, 0.4919f, 0.1856f, 0.0795f, -0.0332f}; - INDArray npdata = Nd4j.createFromArray(data); - HashMap feed_data - = new HashMap() {{ - put("x", npdata); - }}; - List fetch = Arrays.asList("price"); - - Client client = new Client(); - String target = "localhost:9393"; - boolean succ = client.connect(target); - if (succ != true) { - System.out.println("connect failed."); - return ; - } - - Map fetch_map = client.predict(feed_data, fetch); - if (fetch_map == null) { - System.out.println("predict failed."); - return ; - } - - for (Map.Entry e : fetch_map.entrySet()) { - System.out.println("Key = " + e.getKey() + ", Value = " + e.getValue()); - } - return ; - } -} -``` diff --git a/doc/JAVA_SDK_CN.md b/doc/JAVA_SDK_CN.md index f624a4403371f5b284f34cbf310fef64d59602d9..7033b96078e1143567ccb19f14b80fc2b126a45d 100644 --- a/doc/JAVA_SDK_CN.md +++ b/doc/JAVA_SDK_CN.md @@ -19,6 +19,7 @@ Paddle Serving 提供了 Java SDK，支持 Client 端用 Java 语言进行预测 | :---------------------------: | :--------------: | | 0.3.2 | 0.0.1 | +1. 直接使用提供的Java SDK作为Client进行预测 ### 安装 您可以直接下载 jar，安装到本地 Maven 库： @@ -28,14 +29,6 @@ wget https://paddle-serving.bj.bcebos.com/jar/paddle-serving-sdk-java-0.0.1.jar mvn install:install-file -Dfile=$PWD/paddle-serving-sdk-java-0.0.1.jar -DgroupId=io.paddle.serving.client -DartifactId=paddle-serving-sdk-java -Dversion=0.0.1 -Dpackaging=jar ``` -或者从源码进行编译，安装到本地 Maven 库： - -```shell -cd Serving/java -mvn compile -mvn install -``` - ### Maven 配置 ```text @@ -46,64 +39,7 @@ mvn install ``` +2. 从源码进行编译后使用，详细步骤见[文档](../java/README.md). +3. 相关使用示例见[文档](../java/README.md). - -## 使用样例 - -这里将展示如何使用 Java SDK 进行房价预测，更多例子详见 [examples](../java/examples) 文件夹。 - -### 获取房价预测模型 - -```shell -wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz -tar -xzf uci_housing.tar.gz -``` - -### 启动 Python 端 Server - -```shell -python -m paddle_serving_server.serve --model uci_housing_model --port 9393 --use_multilang -``` - -### Client 端代码示例 - -```java -import io.paddle.serving.client.*; -import org.nd4j.linalg.api.ndarray.INDArray; -import org.nd4j.linalg.factory.Nd4j; -import java.util.*; - -public class PaddleServingClientExample { - public static void main( String[] args ) { - float[] data = {0.0137f, -0.1136f, 0.2553f, -0.0692f, - 0.0582f, -0.0727f, -0.1583f, -0.0584f, - 0.6283f, 0.4919f, 0.1856f, 0.0795f, -0.0332f}; - INDArray npdata = Nd4j.createFromArray(data); - HashMap feed_data - = new HashMap() {{ - put("x", npdata); - }}; - List fetch = Arrays.asList("price"); - - Client client = new Client(); - String target = "localhost:9393"; - boolean succ = client.connect(target); - if (succ != true) { - System.out.println("connect failed."); - return ; - } - - Map fetch_map = client.predict(feed_data, fetch); - if (fetch_map == null) { - System.out.println("predict failed."); - return ; - } - - for (Map.Entry e : fetch_map.entrySet()) { - System.out.println("Key = " + e.getKey() + ", Value = " + e.getValue()); - } - return ; - } -} -``` diff --git a/python/paddle_serving_server_gpu/__init__.py b/python/paddle_serving_server_gpu/__init__.py index 9f71138e626bdc3763fd1615abf5f6465cbadbed..d53c67797f66e89e1f6c78d3aeef79d9d8603fc7 100644 --- a/python/paddle_serving_server_gpu/__init__.py +++ b/python/paddle_serving_server_gpu/__init__.py @@ -833,6 +833,7 @@ class MultiLangServer(object): workdir=None, port=9292, device="cpu", + use_encryption_model=False, cube_conf=None): if not self._port_is_available(port): raise SystemExit("Prot {} is already used".format(port)) @@ -847,6 +848,7 @@ class MultiLangServer(object): workdir=workdir, port=self.port_list_[0], device=device, + use_encryption_model=use_encryption_model, cube_conf=cube_conf) self.set_port(port)