Merge branch 'develop' into refine_rpc

c40a009f · MRXLT · GitHub · 8bc9f7f2 · 8a48acd6 · c40a009f
9 changed file
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -51,6 +51,7 @@ option(WITH_MKL	    "Compile Paddle Serving with MKL support."      OFF)
 option(WITH_GPU	    "Compile Paddle Serving with NVIDIA GPU"        OFF)
 option(CLIENT_ONLY  "Compile client libraries and demos only"       OFF)
 option(WITH_ELASTIC_CTR "Compile ELASITC-CTR solution"              OFF)
+option(PACK         "Compile for whl"                               OFF)
 set(WITH_MKLML ${WITH_MKL})
 if (NOT DEFINED WITH_MKLDNN)

--- a/README.md
+++ b/README.md
@@ -5,7 +5,7 @@
 [![License](https://img.shields.io/github/license/PaddlePaddle/Serving)](LICENSE)
 [![Slack](https://img.shields.io/badge/Join-Slack-green)](https://paddleserving.slack.com/archives/CU0PB4K35)
-[中文](./doc/README_CN.md)
+[中文](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md)
 ## Motivation
 Paddle Serving helps deep learning developers deploy an online inference service without much effort. **The goal of this project**: once you have trained a deep neural nets with [Paddle](https://github.com/PaddlePaddle/Paddle), you already have a model inference service. 

--- a/README_CN.md
+++ b/README_CN.md
+<img src='https://paddle-serving.bj.bcebos.com/imdb-demo%2FLogoMakr-3Bd2NM-300dpi.png' width = "600" height = "127">
+[![Release](https://img.shields.io/badge/Release-0.0.3-yellowgreen)](Release)
+[![Issues](https://img.shields.io/github/issues/PaddlePaddle/Serving)](Issues)
+[![License](https://img.shields.io/github/license/PaddlePaddle/Serving)](LICENSE)
+[![Slack](https://img.shields.io/badge/Join-Slack-green)](https://paddleserving.slack.com/archives/CU0PB4K35)
+## 动机
+Paddle Serving 帮助深度学习开发者轻易部署在线预测服务。 **本项目目标**: 只要你使用 [Paddle](https://github.com/PaddlePaddle/Paddle) 训练了一个深度神经网络，你就同时拥有了该模型的预测服务。
+## 核心功能
+- 与Paddle训练紧密连接，绝大部分Paddle模型可以 **一键部署**.
+- 支持 **工业级的服务能力** 例如模型管理，在线加载，在线A/B测试等.
+- 支持 **分布式键值对索引** 助力于大规模稀疏特征作为模型输入.
+- 支持客户端和服务端之间 **高并发和高效通信**.
+- 支持 **多种编程语言** 开发客户端，例如Golang，C++和Python.
+- **可伸缩框架设计** 可支持不限于Paddle的模型服务.
+## 安装
+```shell
+pip install paddle-serving-client
+pip install paddle-serving-server
+```
+## 快速启动示例
+``` shell
+wget --no-check-certificate https://paddle-serving.bj.bcebos.com/uci_housing.tar.gz
+tar -xzf uci_housing.tar.gz
+python -m paddle_serving_server.serve --model uci_housing_model --thread 10 --port 9292
+```
+Python客户端请求
+``` python
+from paddle_serving_client import Client
+client = Client()
+client.load_client_config("uci_housing_client/serving_client_conf.prototxt")
+client.connect(["127.0.0.1:9292"])
+data = [0.0137, -0.1136, 0.2553, -0.0692, 0.0582, -0.0727,
+        -0.1583, -0.0584, 0.6283, 0.4919, 0.1856, 0.0795, -0.0332]
+fetch_map = client.predict(feed={"x": data}, fetch=["price"])
+print(fetch_map)
+```
+## 文档
+[开发文档](doc/DESIGN.md)
+[如何在服务器端配置本地Op?](doc/SERVER_DAG.md)
+[如何开发一个新的Op?](doc/NEW_OPERATOR.md)
+[Golang 客户端](doc/IMDB_GO_CLIENT.md)
+[从源码编译](doc/COMPILE.md)
+[常见问答](doc/FAQ.md)
+## 加入社区
+如果您想要联系其他用户和开发者，欢迎加入我们的 [Slack channel](https://paddleserving.slack.com/archives/CUBPKHKMJ)
+## 如何贡献代码
+如果您想要贡献代码给Paddle Serving，请参考[Contribution Guidelines](doc/CONTRIBUTE.md)
--- a/core/util/src/timer.cc
+++ b/core/util/src/timer.cc
@@ -12,8 +12,8 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License. */
-#include <sys/time.h>
 #include "core/util/include/timer.h"
+#include <sys/time.h>
 namespace baidu {
 namespace paddle_serving {
@@ -56,7 +56,7 @@ double Timer::ElapsedSec() { return _elapsed / 1000000.0; }
 int64_t Timer::TimeStampUS() {
  gettimeofday(&_now, NULL);
-  return _now.tv_usec;
+  return _now.tv_sec * 1000 * 1000L + _now.tv_usec;
 }
 int64_t Timer::Tickus() {

--- a/doc/DESIGN.md
+++ b/doc/DESIGN.md
@@ -47,7 +47,7 @@ Server Python API主要负责加载预估模型，以及生成Paddle Serving需
 ### 3.5 Client Interface
-![Client Interface](client_interface.png)
+<img src='client_inferface.png' width = "600" height = "200">
 ### 3.6 训练过程中使用的Client io

--- a/doc/README_EN.md
+++ b/doc/README_EN.md
-# Paddle Serving
-Paddle Serving is the online inference service framework of [Paddle](https://github.com/PaddlePaddle/Paddle) that can help developers easily deploy a deep learning model service on server side and send request from mobile devices, edge devices as well as data centers. Currently, Paddle Serving supports the deep learning models produced by Paddle althought it can be very easy to support other deep learning framework's model inference. Paddle Serving is designed oriented from industrial practice. For example, multiple models management for online service, double buffers model loading, models online A/B testing are supported. Highly concurrent [Baidu-rpc](https://github.com/apache/incubator-brpc) is used as the underlying communication library which is also from industry practice. Paddle Serving provides user-friendly API that can integrate with Paddle training code seamlessly, and users can finish model training and model serving in an end-to-end fasion.
-## Quick Start
-Paddle Serving supports light-weighted Python API for model inference and can be integrated with trainining process seemlessly. Here is a Boston House Pricing example for users to do quick start.
-### Installation
-```shell
-pip install paddle-serving-client
-pip install paddle-serving-server
-```
-### Training Scripts
-``` python
-import sys
-import paddle
-import paddle.fluid as fluid
-train_reader = paddle.batch(paddle.reader.shuffle(
-    paddle.dataset.uci_housing.train(), buf_size=500), batch_size=16)
-test_reader = paddle.batch(paddle.reader.shuffle(
-    paddle.dataset.uci_housing.test(), buf_size=500), batch_size=16)
-x = fluid.data(name='x', shape=[None, 13], dtype='float32')
-y = fluid.data(name='y', shape=[None, 1], dtype='float32')
-y_predict = fluid.layers.fc(input=x, size=1, act=None)
-cost = fluid.layers.square_error_cost(input=y_predict, label=y)
-avg_loss = fluid.layers.mean(cost)
-sgd_optimizer = fluid.optimizer.SGD(learning_rate=0.01)
-sgd_optimizer.minimize(avg_loss)
-place = fluid.CPUPlace()
-feeder = fluid.DataFeeder(place=place, feed_list=[x, y])
-exe = fluid.Executor(place)
-exe.run(fluid.default_startup_program())
-import paddle_serving_client.io as serving_io
-for pass_id in range(30):
-    for data_train in train_reader():
-        avg_loss_value, = exe.run(
-            fluid.default_main_program(),
-            feed=feeder.feed(data_train),
-            fetch_list=[avg_loss])
-serving_io.save_model(
-    "serving_server_model", "serving_client_conf",
-    {"x": x}, {"y": y_predict}, fluid.default_main_program())
-```
-### Server Side Scripts
-```
-import sys
-from paddle_serving.serving_server import OpMaker
-from paddle_serving.serving_server import OpSeqMaker
-from paddle_serving.serving_server import Server
-op_maker = OpMaker()
-read_op = op_maker.create('general_reader')
-general_infer_op = op_maker.create('general_infer')
-op_seq_maker = OpSeqMaker()
-op_seq_maker.add_op(read_op)
-op_seq_maker.add_op(general_infer_op)
-server = Server()
-server.set_op_sequence(op_seq_maker.get_op_sequence())
-server.load_model_config(sys.argv[1])
-server.prepare_server(workdir="work_dir1", port=9393, device="cpu")
-```
-### Start Server
-```
-python test_server.py serving_server_model
-```
-### Client Side Scripts
-```
-from paddle_serving_client import Client
-import paddle
-import sys
-client = Client()
-client.load_client_config(sys.argv[1])
-client.connect(["127.0.0.1:9292"])
-test_reader = paddle.batch(paddle.reader.shuffle(
-    paddle.dataset.uci_housing.test(), buf_size=500), batch_size=1)
-for data in test_reader():
-    fetch_map = client.predict(feed={"x": data[0][0]}, fetch=["y"])
-    print("{} {}".format(fetch_map["y"][0], data[0][1][0]))
-```
-### Document
-[Design Doc(Chinese)](doc/DESIGN.md)
-[FAQ(Chinese)](doc/FAQ.md)
-### Advanced features and development
-[Develop a serving application with C++(Chinese)](doc/CREATING.md)
-[Compile from source code(Chinese)](doc/INSTALL.md)
-## Contribution
-If you want to contribute code to Paddle Serving, please reference [Contribution Guidelines](
--- a/python/paddle_serving_server/__init__.py
+++ b/python/paddle_serving_server/__init__.py
@@ -110,9 +110,10 @@ class Server(object):
    def set_memory_optimize(self, flag=False):
        self.memory_optimization = flag
-    def set_local_bin(self, path):
+    def check_local_bin(self):
-        self.use_local_bin = True
+        if "SERVING_BIN" in os.environ:
-        self.bin_path = path
+            self.use_local_bin = True
+            self.bin_path = os.environ["SERVING_BIN"]
    def _prepare_engine(self, model_config_path, device):
        if self.model_toolkit_conf == None:
@@ -259,10 +260,11 @@ class Server(object):
    def run_server(self):
        # just run server with system command
        # currently we do not load cube
+        self.check_local_bin()
        if not self.use_local_bin:
            self.download_bin()
        else:
-            print("Use local bin")
+            print("Use local bin : {}".format(self.bin_path))
        command = "{} " \
                  "-enable_model_toolkit " \
                  "-inferservice_path {} " \

--- a/python/paddle_serving_server_gpu/__init__.py
+++ b/python/paddle_serving_server_gpu/__init__.py
@@ -109,9 +109,10 @@ class Server(object):
    def set_memory_optimize(self, flag=False):
        self.memory_optimization = flag
-    def set_local_bin(self, path):
+    def check_local_bin(self):
-        self.use_local_bin = True
+        if "SERVING_BIN" in os.environ:
-        self.bin_path = path
+            self.use_local_bin = True
+            self.bin_path = os.environ["SERVING_BIN"]
    def set_gpuid(self, gpuid=0):
        self.gpuid = gpuid
@@ -243,8 +244,11 @@ class Server(object):
    def run_server(self):
        # just run server with system command
        # currently we do not load cube
+        self.check_local_bin()
        if not self.use_local_bin:
            self.download_bin()
+        else:
+            print("Use local bin : {}".format(self.bin_path))
        command = "{} " \
                  "-enable_model_toolkit " \
                  "-inferservice_path {} " \

--- a/python/setup.py.client.in
+++ b/python/setup.py.client.in
@@ -35,7 +35,10 @@ def copy_lib():
        os.popen('cp {} ./paddle_serving_client/lib'.format(text.strip().split(' ')[1]))
 max_version, mid_version, min_version = python_version()
-#copy_lib()
+if '${PACK}' == 'ON':
+    copy_lib()
 REQUIRED_PACKAGES = [
    'six >= 1.10.0', 'protobuf >= 3.1.0','paddlepaddle'
@@ -91,4 +94,3 @@ setup(
    ],
    license='Apache 2.0',
    keywords=('paddle-serving serving-client deployment industrial easy-to-use'))