diff --git a/doc/COMPILE.md b/doc/COMPILE.md index 03d135e006c36e57e52b1d353c79217b53baa5e1..ec1900a89b9b2f0112f0d44adc539b138438bba7 100644 --- a/doc/COMPILE.md +++ b/doc/COMPILE.md @@ -6,12 +6,11 @@ | module | version | | :--------------------------: | :-------------------------------: | -| OS | CentOS 7 | -| gcc | 4.8.5 and later | -| gcc-c++ | 4.8.5 and later | -| git | 3.82 and later | +| OS | Ubuntu16 and 18/CentOS 7 | +| gcc | 4.8.5(Cuda 9.0 and 10.0) and 8.2(Others) | +| gcc-c++ | 4.8.5(Cuda 9.0 and 10.0) and 8.2(Others) | | cmake | 3.2.0 and later | -| Python | 2.7.2 and later / 3.6 and later | +| Python | 2.7.2 and later / 3.5.1 and later | | Go | 1.9.2 and later | | git | 2.17.1 and later | | glibc-static | 2.17 | @@ -19,19 +18,13 @@ | bzip2-devel | 1.0.6 and later | | python-devel / python3-devel | 2.7.5 and later / 3.6.8 and later | | sqlite-devel | 3.7.17 and later | -| patchelf | 0.9 and later | +| patchelf | 0.9 | | libXext | 1.3.3 | | libSM | 1.2.2 | | libXrender | 0.9.10 | It is recommended to use Docker for compilation. We have prepared the Paddle Serving compilation environment for you, see [this document](DOCKER_IMAGES.md). -This document will take Python2 as an example to show how to compile Paddle Serving. If you want to compile with Python3, just adjust the Python options of cmake: - -- Set `DPYTHON_INCLUDE_DIR` to `$PYTHONROOT/include/python3.6m/` -- Set `DPYTHON_LIBRARIES` to `$PYTHONROOT/lib64/libpython3.6.so` -- Set `DPYTHON_EXECUTABLE` to `$PYTHONROOT/bin/python3.6` - ## Get Code ``` python @@ -39,19 +32,47 @@ git clone https://github.com/PaddlePaddle/Serving cd Serving && git submodule update --init --recursive ``` - - - -## PYTHONROOT Setting +## PYTHONROOT settings ```shell -# for example, the path of python is /usr/bin/python, you can set /usr as PYTHONROOT -export PYTHONROOT=/usr/ +# For example, the path of python is /usr/bin/python, you can set PYTHONROOT +export PYTHONROOT=/usr ``` -In the default centos7 image we provide, the Python path is `/usr/bin/python`. If you want to use our centos6 image, you need to set it to `export PYTHONROOT=/usr/local/python2.7/`. +If you are using a Docker development image, please follow the following to determine the Python version to be compiled, and set the corresponding environment variables +``` +#Python 2.7 +export PYTHONROOT=/usr/local/python2.7.15/ +export PYTHON_INCLUDE_DIR=$PYTHONROOT/include/python2.7/ +export PYTHON_LIBRARIES=$PYTHONROOT/lib/libpython2.7.so +export PYTHON_EXECUTABLE=$PYTHONROOT/bin/python2.7 + +#Python 3.5 +export PYTHONROOT=/usr/local/python3.5.1 +export PYTHON_INCLUDE_DIR=$PYTHONROOT/include/python3.5m +export PYTHON_LIBRARIES=$PYTHONROOT/lib/libpython3.5m.so +export PYTHON_EXECUTABLE=$PYTHONROOT/bin/python3.5 + +#Python3.6 +export PYTHONROOT=/usr/local/ +export PYTHON_INCLUDE_DIR=$PYTHONROOT/include/python3.6m +export PYTHON_LIBRARIES=$PYTHONROOT/lib/libpython3.6m.so +export PYTHON_EXECUTABLE=$PYTHONROOT/bin/python3.6 + +#Python3.7 +export PYTHONROOT=/usr/local/ +export PYTHON_INCLUDE_DIR=$PYTHONROOT/include/python3.7m +export PYTHON_LIBRARIES=$PYTHONROOT/lib/libpython3.7m.so +export PYTHON_EXECUTABLE=$PYTHONROOT/bin/python3.7 + +#Python3.8 +export PYTHONROOT=/usr/local/ +export PYTHON_INCLUDE_DIR=$PYTHONROOT/include/python3.8 +export PYTHON_LIBRARIES=$PYTHONROOT/lib/libpython3.8.so +export PYTHON_EXECUTABLE=$PYTHONROOT/bin/python3.8 +``` ## Install Python dependencies @@ -59,14 +80,11 @@ In the default centos7 image we provide, the Python path is `/usr/bin/python`. I pip install -r python/requirements.txt ``` -If Python3 is used, replace `pip` with `pip3`. +If you use other Python version, please use the right `pip` accordingly. ## GOPATH Setting +The default GOPATH is set to `$HOME/go`, you can also set it to other values. **If it is the Docker environment provided by Serving, you do not need to set up.** - -## Compile Arguments - -The default GOPATH is `$HOME/go`, which you can set to other values. ```shell export GOPATH=$HOME/go export PATH=$PATH:$GOPATH/bin @@ -100,52 +118,42 @@ make -j10 you can execute `make install` to put targets under directory `./output`, you need to add`-DCMAKE_INSTALL_PREFIX=./output`to specify output path to cmake command shown above. ### Integrated GPU version paddle inference library -### CUDA_PATH is the cuda install path,use the command(whereis cuda) to check,it should be /usr/local/cuda. -### CUDNN_LIBRARY && CUDA_CUDART_LIBRARY is the lib path, it should be /usr/local/cuda/lib64/ - -``` shell -export CUDA_PATH='/usr/local/cuda' -export CUDNN_LIBRARY='/usr/local/cuda/lib64/' -export CUDA_CUDART_LIBRARY="/usr/local/cuda/lib64/" -mkdir server-build-gpu && cd server-build-gpu -cmake -DPYTHON_INCLUDE_DIR=$PYTHONROOT/include/python2.7/ \ - -DPYTHON_LIBRARIES=$PYTHONROOT/lib/libpython2.7.so \ - -DPYTHON_EXECUTABLE=$PYTHONROOT/bin/python \ - -DCUDA_TOOLKIT_ROOT_DIR=${CUDA_PATH} \ - -DCUDNN_LIBRARY=${CUDNN_LIBRARY} \ - -DCUDA_CUDART_LIBRARY=${CUDA_CUDART_LIBRARY} \ - -DSERVER=ON \ - -DWITH_GPU=ON .. -make -j10 -``` +Compared with CPU environment, GPU environment needs to refer to the following table, +**It should be noted that the following table is used as a reference for non-Docker compilation environment. The Docker compilation environment has been configured with relevant parameters and does not need to be specified in cmake process. ** -### Integrated TRT version paddle inference library +| cmake environment variable | meaning | GPU environment considerations | whether Docker environment is needed | +|-----------------------|------------------------- ------------|-------------------------------|----- ---------------| +| CUDA_TOOLKIT_ROOT_DIR | cuda installation path, usually /usr/local/cuda | Required for all environments | No +(/usr/local/cuda) | +| CUDNN_LIBRARY | The directory where libcudnn.so.* is located, usually /usr/local/cuda/lib64/ | Required for all environments | No (/usr/local/cuda/lib64/) | +| CUDA_CUDART_LIBRARY | The directory where libcudart.so.* is located, usually /usr/local/cuda/lib64/ | Required for all environments | No (/usr/local/cuda/lib64/) | +| TENSORRT_ROOT | The upper level directory of the directory where libnvinfer.so.* is located, depends on the TensorRT installation directory | Cuda 9.0/10.0 does not need, other needs | No (/usr) | -``` +If not in Docker environment, users can refer to the following execution methods. The specific path is subject to the current environment, and the code is only for reference. + +``` shell export CUDA_PATH='/usr/local/cuda' export CUDNN_LIBRARY='/usr/local/cuda/lib64/' export CUDA_CUDART_LIBRARY="/usr/local/cuda/lib64/" +export TENSORRT_LIBRARY_PATH="/usr/local/TensorRT-6.0.1.5/targets/x86_64-linux-gnu/" -mkdir server-build-trt && cd server-build-trt -cmake -DPYTHON_INCLUDE_DIR=$PYTHONROOT/include/python2.7/ \ - -DPYTHON_LIBRARIES=$PYTHONROOT/lib/libpython2.7.so \ - -DPYTHON_EXECUTABLE=$PYTHONROOT/bin/python \ - -DTENSORRT_ROOT=${TENSORRT_LIBRARY_PATH} \ +mkdir server-build-gpu && cd server-build-gpu +cmake -DPYTHON_INCLUDE_DIR=$PYTHON_INCLUDE_DIR \ + -DPYTHON_LIBRARIES=$PYTHON_LIBRARIES \ + -DPYTHON_EXECUTABLE=$PYTHON_EXECUTABLE \ -DCUDA_TOOLKIT_ROOT_DIR=${CUDA_PATH} \ -DCUDNN_LIBRARY=${CUDNN_LIBRARY} \ -DCUDA_CUDART_LIBRARY=${CUDA_CUDART_LIBRARY} \ + -DTENSORRT_ROOT=${TENSORRT_LIBRARY_PATH} -DSERVER=ON \ - -DWITH_GPU=ON \ - -DWITH_TRT=ON .. + -DWITH_GPU=ON .. make -j10 ``` -execute `make install` to put targets under directory `./output` - -**Attention：** After the compilation is successful, you need to set the path of `SERVING_BIN`. See [Note](https://github.com/PaddlePaddle/Serving/blob/develop/doc/COMPILE.md#Note) for details. - +Execute `make install` to put the target output in the `./output` directory. +**Note:** After the compilation is successful, you need to set the `SERVING_BIN` path, see the following [Notes](COMPILE.md#Notes) ). ## Compile Client @@ -208,7 +216,6 @@ Please use the example under `python/examples` to verify. | CLIENT | Compile Paddle Serving Client | OFF | | SERVER | Compile Paddle Serving Server | OFF | | APP | Compile Paddle Serving App package | OFF | -| WITH_ELASTIC_CTR | Compile ELASITC-CTR solution | OFF | | PACK | Compile for whl | OFF | ### WITH_GPU Option @@ -230,11 +237,13 @@ Note here: The following is the base library version matching relationship used by the PaddlePaddle release version for reference: -| | CUDA | CuDNN | TensorRT | -| :----: | :-----: | :----------------------: | :----: | -| post9 | 9.0 | CuDNN 7.3.1 for CUDA 9.0 | | -| post10 | 10.0 | CuDNN 7.5.1 for CUDA 10.0| | -| trt | 10.1 | CuDNN 7.5.1 for CUDA 10.1| 6.0.1.5 | +| | CUDA | CuDNN | TensorRT | +| :----: | :-----: | :----------: | :----: | +| post9 | 9.0 | CuDNN 7.6.4 | | +| post10 | 10.0 | CuDNN 7.6.5 | | +| post101 | 10.1 | CuDNN 7.6.5 | 6.0.1 | +| post102 | 10.2 | CuDNN 8.0.5 | 7.1.3 | +| post11 | 11.0 | CuDNN 8.0.4 | 7.1.3 | ### How to make the compiler detect the CuDNN library diff --git a/doc/DOCKER_IMAGES.md b/doc/DOCKER_IMAGES.md index 2de493fa6fcb20deb0601a2a338436785e605662..54cbe2ef94c980fec3dfd823deeccd4358ebc813 100644 --- a/doc/DOCKER_IMAGES.md +++ b/doc/DOCKER_IMAGES.md @@ -8,11 +8,10 @@ This document maintains a list of docker images provided by Paddle Serving. You can get images in two ways: -1. Pull image directly from `hub.baidubce.com ` or `docker.io` through TAG: +1. Pull image directly from `registry.baidubce.com ` through TAG: ```shell - docker pull hub.baidubce.com/paddlepaddle/serving: # hub.baidubce.com - docker pull paddlepaddle/serving: # hub.docker.com + docker pull registry.baidubce.com/paddlepaddle/serving: # registry.baidubce.com ``` 2. Building image based on dockerfile @@ -20,7 +19,7 @@ You can get images in two ways: Create a new folder and copy Dockerfile to this folder, and run the following command: ```shell - docker build -t : . + docker build -f ${DOCKERFILE} -t : . ``` @@ -59,6 +58,42 @@ hub.baidubce.com/paddlepaddle/serving:xpu-beta Running a CUDA container requires a machine with at least one CUDA-capable GPU and a driver compatible with the CUDA toolkit version you are using. -The machine running the CUDA container **only requires the NVIDIA driver**, the CUDA toolkit doesn't have to be installed. +The machine running the CUDA container **only requires the NVIDIA driver**, the CUDA toolkit does not have to be installed. For the relationship between CUDA toolkit version, Driver version and GPU architecture, please refer to [nvidia-docker wiki](https://github.com/NVIDIA/nvidia-docker/wiki/CUDA). + +# (Attachment) The List of All the Docker images + +Develop Images: + +| Env | Version | Docker images tag | OS | Gcc Version | +|----------|---------|------------------------------|-----------|-------------| +| CPU | 0.5.0 | 0.5.0-devel | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | 0.4.0-devel | CentOS 7 | 4.8.5 | +| Cuda9.0 | 0.5.0 | 0.5.0-cuda9.0-cudnn7-devel | Ubuntu 16 | 4.8.5 | +| | <=0.4.0 | 0.4.0-cuda9.0-cudnn7-devel | CentOS 7 | 4.8.5 | +| Cuda10.0 | 0.5.0 | 0.5.0-cuda10.0-cudnn7-devel | Ubuntu 16 | 4.8.5 | +| | <=0.4.0 | 0.4.0-cuda10.0-cudnn7-devel | CentOS 7 | 4.8.5 | +| Cuda10.1 | 0.5.0 | 0.5.0-cuda10.1-cudnn7-devel | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | 0.4.0-cuda10.1-cudnn7-devel | CentOS 7 | 4.8.5 | +| Cuda10.2 | 0.5.0 | 0.5.0-cuda10.2-cudnn8-devel | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | Nan | Nan | Nan | +| Cuda11.0 | 0.5.0 | 0.5.0-cuda11.0-cudnn8-devel | Ubuntu 18 | 8.2.0 | +| | <=0.4.0 | Nan | Nan | Nan | + +Running Images: + +| Env | Version | Docker images tag | OS | Gcc Version | +|----------|---------|-----------------------|-----------|-------------| +| CPU | 0.5.0 | 0.5.0 | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | 0.4.0 | CentOS 7 | 4.8.5 | +| Cuda9.0 | 0.5.0 | 0.5.0-cuda9.0-cudnn7 | Ubuntu 16 | 4.8.5 | +| | <=0.4.0 | 0.4.0-cuda9.0-cudnn7 | CentOS 7 | 4.8.5 | +| Cuda10.0 | 0.5.0 | 0.5.0-cuda10.0-cudnn7 | Ubuntu 16 | 4.8.5 | +| | <=0.4.0 | 0.4.0-cuda10.0-cudnn7 | CentOS 7 | 4.8.5 | +| Cuda10.1 | 0.5.0 | 0.5.0-cuda10.1-cudnn7 | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | 0.4.0-cuda10.1-cudnn7 | CentOS 7 | 4.8.5 | +| Cuda10.2 | 0.5.0 | 0.5.0-cuda10.2-cudnn8 | Ubuntu 16 | 8.2.0 | +| | <=0.4.0 | Nan | Nan | Nan | +| Cuda11.0 | 0.5.0 | 0.5.0-cuda11.0-cudnn8 | Ubuntu 18 | 8.2.0 | +| | <=0.4.0 | Nan | Nan | Nan | diff --git a/doc/DOCKER_IMAGES_CN.md b/doc/DOCKER_IMAGES_CN.md index a06d715d362c0ad6881a24c0e9162de64966a599..fc6e6e3b879751b52b1491bbd666749bb1243450 100644 --- a/doc/DOCKER_IMAGES_CN.md +++ b/doc/DOCKER_IMAGES_CN.md @@ -8,10 +8,10 @@ 您可以通过两种方式获取镜像。 -1. 通过 TAG 直接从 `hub.baidubce.com ` 或 拉取镜像，具体TAG请参见下文的**镜像说明**章节的表格。 +1. 通过 TAG 直接从 `registry.baidubce.com ` 或 拉取镜像，具体TAG请参见下文的**镜像说明**章节的表格。 ```shell - docker pull registry.baidubce.com/paddlepaddle/serving: # hub.baidubce.com + docker pull registry.baidubce.com/paddlepaddle/serving: # registry.baidubce.com ``` 2. 基于 Dockerfile 构建镜像 diff --git a/doc/RUN_IN_DOCKER.md b/doc/RUN_IN_DOCKER.md index caffcf3b8ea5fec820c02f1ed5e02b45c4417d93..335ff8a2cd76762883154a1b0decdd6edc68cad1 100644 --- a/doc/RUN_IN_DOCKER.md +++ b/doc/RUN_IN_DOCKER.md @@ -17,14 +17,14 @@ This document takes Python2 as an example to show how to run Paddle Serving in d Refer to [this document](DOCKER_IMAGES.md) for a docker image: ```shell -docker pull hub.baidubce.com/paddlepaddle/serving:latest +docker pull hub.baidubce.com/paddlepaddle/serving:latest-devel ``` ### Create container ```bash -docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest +docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-devel docker exec -it test bash ``` @@ -46,20 +46,20 @@ The GPU version is basically the same as the CPU version, with only some differe Refer to [this document](DOCKER_IMAGES.md) for a docker image, the following is an example of an `cuda9.0-cudnn7` image: ```shell -docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda10.2-cudnn8-devel ``` ### Create container ```bash -nvidia-docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +nvidia-docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda10.2-cudnn8-devel nvidia-docker exec -it test bash ``` or ```bash -docker run --gpus all -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda9.0-cudnn7 +docker run --gpus all -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-cuda10.2-cudnn8-devel docker exec -it test bash ``` @@ -69,7 +69,7 @@ The `-p` option is to map the `9292` port of the container to the `9292` port of The mirror comes with `paddle_serving_server_gpu`, `paddle_serving_client`, and `paddle_serving_app` corresponding to the mirror tag version. If users don’t need to change the version, they can use it directly, which is suitable for environments without extranet services. -If you need to change the version, please refer to the instructions on the homepage to download the pip package of the corresponding version. +If you need to change the version, please refer to the instructions on the homepage to download the pip package of the corresponding version. [LATEST_PACKAGES](./LATEST_PACKAGES.md) ## Precautious diff --git a/doc/RUN_IN_DOCKER_CN.md b/doc/RUN_IN_DOCKER_CN.md index 436ea7dab1643d104664346665b719542e9df7e1..2f083fb7c07b7cdcfa06eaa99a0635efefa4287e 100644 --- a/doc/RUN_IN_DOCKER_CN.md +++ b/doc/RUN_IN_DOCKER_CN.md @@ -25,7 +25,7 @@ docker pull hub.baidubce.com/paddlepaddle/serving:latest-devel ### 创建容器并进入 ```bash -docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest +docker run -p 9292:9292 --name test -dit hub.baidubce.com/paddlepaddle/serving:latest-devel docker exec -it test bash ``` @@ -39,6 +39,10 @@ docker exec -it test bash ## GPU 版本 +```shell +docker pull hub.baidubce.com/paddlepaddle/serving:latest-cuda10.2-cudnn8-devel +``` + ### 创建容器并进入 ```bash @@ -57,7 +61,7 @@ docker exec -it test bash 镜像里自带对应镜像tag版本的`paddle_serving_server_gpu`，`paddle_serving_client`，`paddle_serving_app`，如果用户不需要更改版本，可以直接使用，适用于没有外网服务的环境。 -如果需要更换版本，请参照首页的指导，下载对应版本的pip包。[最新安装包合集](https://github.com/wangjiawei04/Serving/blob/docs_0.5.0/doc/LATEST_PACKAGES.md) +如果需要更换版本，请参照首页的指导，下载对应版本的pip包。[最新安装包合集](LATEST_PACKAGES.md) ## 注意事项 diff --git a/doc/SAVE.md b/doc/SAVE.md index 8a909dc98d60579cd2861f5cdf38619264bae2fa..32562fa55af253bdaa6328c9bd02f5d54328161b 100644 --- a/doc/SAVE.md +++ b/doc/SAVE.md @@ -2,7 +2,74 @@ ([简体中文](./SAVE_CN.md)|English) -## Save from training or prediction script +## Export from saved model files + +you can use a build-in python module called `paddle_serving_client.convert` to convert it. +```python +python -m paddle_serving_client.convert --dirname ./your_inference_model_dir +``` + +If you have saved model files using Paddle's `save_inference_model` API, you can use Paddle Serving's` inference_model_to_serving` API to convert it into a model file that can be used for Paddle Serving. +```python +import paddle_serving_client.io as serving_io +serving_io.inference_model_to_serving(dirname, serving_server="serving_server", serving_client="serving_client", model_filename=None, params_filename=None ) +``` +Arguments are the same as `inference_model_to_serving` API. +| Argument | Type | Default | Description | +|--------------|------|-----------|--------------------------------| +| `dirname` | str | - | Path of saved model files. Program file and parameter files are saved in this directory. | +| `serving_server` | str | `"serving_server"` | The path of model files and configuration files for server. | +| `serving_client` | str | `"serving_client"` | The path of configuration files for client. | +| `model_filename` | str | None | The name of file to load the inference program. If it is None, the default filename `__model__` will be used. | +| `params_filename` | str | None | The name of file to load all parameters. It is only used for the case that all parameters were saved in a single binary file. If parameters were saved in separate files, set it as None. | + +**Demo: Convert From Dynamic Graph** + +PaddlePaddle 2.0 provides a new dynamic graph mode, so here we use imagenet ResNet50 dynamic graph as an example to teach how to export from a saved model and use it for real online inference scenarios. + +``` +wget https://paddle-serving.bj.bcebos.com/others/dygraph_res50.tar #模型 +wget https://paddle-serving.bj.bcebos.com/imagenet-example/daisy.jpg #示例输入（向日葵） +tar xf dygraph_res50.tar +python -m paddle_serving_client.convert --dirname . --model_filename dygraph_model.pdmodel --params_filename dygraph_model.pdiparams --serving_server serving_server --serving_client serving_client``` + +We can see that the `serving_server` and `serving_client` folders hold the server and client configuration of the model respectively + +Start the server (GPU) + +``` +python -m paddle_serving_server_gpu.serve --model serving_server --port 9393 --gpu_id 0 +``` + +Client (`test_client.py`) +``` +from paddle_serving_client import Client +from paddle_serving_app.reader import Sequential, File2Image, Resize, CenterCrop +from paddle_serving_app.reader import RGB2BGR, Transpose, Div, Normalize + +client = Client() +client.load_client_config( + "serving_client/serving_client_conf.prototxt") +client.connect(["127.0.0.1:9393"]) + +seq = Sequential([ + File2Image(), Resize(256), CenterCrop(224), RGB2BGR(), Transpose((2, 0, 1)), + Div(255), Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225], True) +]) + +image_file = "daisy.jpg" +img = seq(image_file) +fetch_map = client.predict(feed={"inputs": img}, fetch=["save_infer_model/scale_0.tmp_0"]) +print(fetch_map["save_infer_model/scale_0.tmp_0"].reshape(-1)) +``` +Run +``` +python test_client.py +``` + +You can see that the prediction has been successfully executed. The above is the content predicted by the dynamic graph ResNet50 model on Serving. The use of other dynamic graph models is similar. + +## Save from training or prediction script (Static Graph Mode) Currently, paddle serving provides a save_model interface for users to access, the interface is similar with `save_inference_model` of Paddle. ``` python import paddle_serving_client.io as serving_io @@ -31,22 +98,3 @@ for line in sys.stdin: fetch_map = client.predict(feed=feed, fetch=fetch) print("{} {}".format(fetch_map["prediction"][1], label[0])) ``` - -## Export from saved model files -If you have saved model files using Paddle's `save_inference_model` API, you can use Paddle Serving's` inference_model_to_serving` API to convert it into a model file that can be used for Paddle Serving. -```python -import paddle_serving_client.io as serving_io -serving_io.inference_model_to_serving(dirname, serving_server="serving_server", serving_client="serving_client", model_filename=None, params_filename=None ) -``` -Or you can use a build-in python module called `paddle_serving_client.convert` to convert it. -```python -python -m paddle_serving_client.convert --dirname ./your_inference_model_dir -``` -Arguments are the same as `inference_model_to_serving` API. -| Argument | Type | Default | Description | -|--------------|------|-----------|--------------------------------| -| `dirname` | str | - | Path of saved model files. Program file and parameter files are saved in this directory. | -| `serving_server` | str | `"serving_server"` | The path of model files and configuration files for server. | -| `serving_client` | str | `"serving_client"` | The path of configuration files for client. | -| `model_filename` | str | None | The name of file to load the inference program. If it is None, the default filename `__model__` will be used. | -| `params_filename` | str | None | The name of file to load all parameters. It is only used for the case that all parameters were saved in a single binary file. If parameters were saved in separate files, set it as None. |