diff --git a/doc/BAIDU_KUNLUN_XPU_SERVING.md b/doc/BAIDU_KUNLUN_XPU_SERVING.md index 31a72c9be9da2cd846eafbba34360ae9febfafbc..02568642bad6aafd147b628a1c6607fd8af9fed3 100644 --- a/doc/BAIDU_KUNLUN_XPU_SERVING.md +++ b/doc/BAIDU_KUNLUN_XPU_SERVING.md @@ -1,12 +1,12 @@ # Paddle Serving Using Baidu Kunlun Chips (English|[简体中文](./BAIDU_KUNLUN_XPU_SERVING_CN.md)) -Paddle serving supports deployment using Baidu Kunlun chips. At present, the pilot support is deployed on the ARM server with Baidu Kunlun chips - (such as Phytium FT-2000+/64). We will improve +Paddle serving supports deployment using Baidu Kunlun chips. Currently, it supports deployment on the ARM CPU server with Baidu Kunlun chips + (such as Phytium FT-2000+/64), or Intel CPU with Baidu Kunlun chips. We will improve the deployment capability on various heterogeneous hardware servers in the future. # Compilation and installation -Refer to [compile](COMPILE.md) document to setup the compilation environment。 +Refer to [compile](COMPILE.md) document to setup the compilation environment. The following is based on FeiTeng FT-2000 +/64 platform. ## Compilatiton * Compile the Serving Server ``` @@ -54,11 +54,11 @@ make -j10 ``` ## Install the wheel package After the compilations stages above, the whl package will be generated in ```python/dist/``` under the specific temporary directories. -For example, after the Server Compiation step，the whl package will be produced under the server-build-arm/python/dist directory, and you can run ```pip install -u python/dist/*.whl``` to install the package。 +For example, after the Server Compiation step，the whl package will be produced under the server-build-arm/python/dist directory, and you can run ```pip install -u python/dist/*.whl``` to install the package. # Request parameters description In order to deploy serving - service on the arm server with Baidu Kunlun xpu chips and use the acceleration capability of Paddle-Lite，please specify the following parameters during deployment。 + service on the arm server with Baidu Kunlun xpu chips and use the acceleration capability of Paddle-Lite，please specify the following parameters during deployment. | param | param description | about | | :------- | :------------------------------- | :----------------------------------------------------------------- | | use_lite | using Paddle-Lite Engine | use the inference capability of Paddle-Lite | @@ -72,21 +72,21 @@ tar -xzf uci_housing.tar.gz ``` ## Start RPC service There are mainly three deployment methods： -* deploy on the ARM server with Baidu xpu using the acceleration capability of Paddle-Lite and xpu； -* deploy on the ARM server standalone with Paddle-Lite； -* deploy on the ARM server standalone without Paddle-Lite。 +* deploy on the cpu server with Baidu xpu using the acceleration capability of Paddle-Lite and xpu； +* deploy on the cpu server standalone with Paddle-Lite； +* deploy on the cpu server standalone without Paddle-Lite. -The first two deployment methods are recommended。 +The first two deployment methods are recommended. -Start the rpc service, deploying on ARM server with Baidu Kunlun chips，and accelerate with Paddle-Lite and Baidu Kunlun xpu. +Start the rpc service, deploying on cpu server with Baidu Kunlun chips，and accelerate with Paddle-Lite and Baidu Kunlun xpu. ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 --use_lite --use_xpu --ir_optim ``` -Start the rpc service, deploying on ARM server，and accelerate with Paddle-Lite. +Start the rpc service, deploying on cpu server，and accelerate with Paddle-Lite. ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 --use_lite --ir_optim ``` -Start the rpc service, deploying on ARM server. +Start the rpc service, deploying on cpu server. ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 ``` @@ -102,8 +102,17 @@ data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, fetch_map = client.predict(feed={"x": np.array(data).reshape(1,13,1)}, fetch=["price"]) print(fetch_map) ``` -Some examples are provided below, and other models can be modifed with reference to these examples。 +# Others +## Model example and explanation + +Some examples are provided below, and other models can be modifed with reference to these examples. | sample name | sample links | | :---------- | :---------------------------------------------------------- | | fit_a_line | [fit_a_line_xpu](../python/examples/xpu/fit_a_line_xpu) | | resnet | [resnet_v2_50_xpu](../python/examples/xpu/resnet_v2_50_xpu) | + +Note：Supported model lists refer to [doc](https://paddlelite.paddlepaddle.org.cn/introduction/support_model_list.html). There are differences in the adaptation of different models, and there may be some unsupported cases. If you have any problem，please submit [Github issue](https://github.com/PaddlePaddle/Serving/issues), and we will follow up in real time. + +## Kunlun chip related reference materials +* [PaddlePaddle on Baidu Kunlun xpu chips](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/xpu_docs/index_cn.html) +* [Deployment on Baidu Kunlun xpu chips using PaddleLite](https://paddlelite.paddlepaddle.org.cn/demo_guides/baidu_xpu.html) diff --git a/doc/BAIDU_KUNLUN_XPU_SERVING_CN.md b/doc/BAIDU_KUNLUN_XPU_SERVING_CN.md index d564e4fb736e05ce37dae918a76752bd06b75bef..fb7de26e016388dbcc3e5db23d8232743fdd792e 100644 --- a/doc/BAIDU_KUNLUN_XPU_SERVING_CN.md +++ b/doc/BAIDU_KUNLUN_XPU_SERVING_CN.md @@ -1,10 +1,10 @@ # Paddle Serving使用百度昆仑芯片部署 (简体中文|[English](./BAIDU_KUNLUN_XPU_SERVING.md)) -Paddle Serving支持使用百度昆仑芯片进行预测部署。目前试验性支持在百度昆仑芯片和arm服务器（如飞腾 FT-2000+/64）上进行部署，后续完善对其他异构硬件服务器部署能力。 +Paddle Serving支持使用百度昆仑芯片进行预测部署。目前支持在百度昆仑芯片和arm服务器（如飞腾 FT-2000+/64）, 或者百度昆仑芯片和Intel CPU服务器，上进行部署，后续完善对其他异构硬件服务器部署能力。 # 编译、安装 -基本环境配置可参考[该文档](COMPILE_CN.md)进行配置。 +基本环境配置可参考[该文档](COMPILE_CN.md)进行配置。下面以飞腾FT-2000+/64机器为例进行介绍。 ## 编译 * 编译server部分 ``` @@ -20,7 +20,7 @@ cmake -DPYTHON_INCLUDE_DIR=/usr/include/python3.7m/ \ -DSERVER=ON .. make -j10 ``` -可以执行`make install`把目标产出放在`./output`目录下，cmake阶段需添加`-DCMAKE_INSTALL_PREFIX=./output`选项来指定存放路径。 +可以执行`make install`把目标产出放在`./output`目录下，cmake阶段需添加`-DCMAKE_INSTALL_PREFIX=./output`选项来指定存放路径。在支持AVX2指令集的Intel CPU平台上请指定```-DWITH_MKL=ON```编译选项。 * 编译client部分 ``` mkdir -p client-build-arm && cd client-build-arm @@ -55,11 +55,11 @@ make -j10 # 请求参数说明 为了支持arm+xpu服务部署，使用Paddle-Lite加速能力，请求时需使用以下参数。 -|参数|参数说明|备注| -|:--|:--|:--| -|use_lite|使用Paddle-Lite Engine|使用Paddle-Lite cpu预测能力| -|use_xpu|使用Baidu Kunlun进行预测|该选项需要与use_lite配合使用| -|ir_optim|开启Paddle-Lite计算子图优化|详细见[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite)| +| 参数 | 参数说明 | 备注 | +| :------- | :-------------------------- | :--------------------------------------------------------------- | +| use_lite | 使用Paddle-Lite Engine | 使用Paddle-Lite cpu预测能力 | +| use_xpu | 使用Baidu Kunlun进行预测 | 该选项需要与use_lite配合使用 | +| ir_optim | 开启Paddle-Lite计算子图优化 | 详细见[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) | # 部署使用示例 ## 下载模型 ``` @@ -68,21 +68,21 @@ tar -xzf uci_housing.tar.gz ``` ## 启动rpc服务 主要有三种启动配置： -* 使用arm cpu+xpu部署，使用Paddle-Lite xpu优化加速能力； -* 单独使用arm cpu部署，使用Paddle-Lite优化加速能力； -* 使用arm cpu部署，不使用Paddle-Lite加速。 +* 使用cpu+xpu部署，使用Paddle-Lite xpu优化加速能力； +* 单独使用cpu部署，使用Paddle-Lite优化加速能力； +* 使用cpu部署，不使用Paddle-Lite加速。 推荐使用前两种部署方式。 -启动rpc服务，使用arm cpu+xpu部署，使用Paddle-Lite xpu优化加速能力 +启动rpc服务，使用cpu+xpu部署，使用Paddle-Lite xpu优化加速能力 ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 --use_lite --use_xpu --ir_optim ``` -启动rpc服务，使用arm cpu部署, 使用Paddle-Lite加速能力 +启动rpc服务，使用cpu部署, 使用Paddle-Lite加速能力 ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 --use_lite --ir_optim ``` -启动rpc服务，使用arm cpu部署, 不使用Paddle-Lite加速能力 +启动rpc服务，使用cpu部署, 不使用Paddle-Lite加速能力 ``` python3 -m paddle_serving_server.serve --model uci_housing_model --thread 6 --port 9292 ``` @@ -98,8 +98,16 @@ data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727, fetch_map = client.predict(feed={"x": np.array(data).reshape(1,13,1)}, fetch=["price"]) print(fetch_map) ``` +# 其他说明 + +## 模型实例及说明 以下提供部分样例，其他模型可参照进行修改。 -|示例名称|示例链接| -|:-----|:--| -|fit_a_line|[fit_a_line_xpu](../python/examples/xpu/fit_a_line_xpu)| -|resnet|[resnet_v2_50_xpu](../python/examples/xpu/resnet_v2_50_xpu)| +| 示例名称 | 示例链接 | +| :--------- | :---------------------------------------------------------- | +| fit_a_line | [fit_a_line_xpu](../python/examples/xpu/fit_a_line_xpu) | +| resnet | [resnet_v2_50_xpu](../python/examples/xpu/resnet_v2_50_xpu) | + +注：支持昆仑芯片部署模型列表见[链接](https://paddlelite.paddlepaddle.org.cn/introduction/support_model_list.html)。不同模型适配上存在差异，可能存在不支持的情况，部署使用存在问题时，欢迎以[Github issue](https://github.com/PaddlePaddle/Serving/issues)，我们会实时跟进。 +## 昆仑芯片支持相关参考资料 +* [昆仑XPU芯片运行飞桨](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/guides/xpu_docs/index_cn.html) +* [PaddleLite使用百度XPU预测部署](https://paddlelite.paddlepaddle.org.cn/demo_guides/baidu_xpu.html)