Merge branch 'develop' into pipeline-auto-batch

doc/COMPILE.md

@@ -23,7 +23,7 @@
 |           libXext            |                            1.3.3                             |
 |            libSM             |                            1.2.2                             |
 |          libXrender          |                            0.9.10                            |
-|          python-whl          | numpy>=1.12, <=1.16.4<br/>google>=2.0.3<br/>protobuf>=3.12.2<br/>grpcio-tools>=1.28.1<br/>grpcio>=1.28.1<br/>func-timeout>=4.3.5<br/>pyyaml>=1.3.0<br/>sentencepiece==0.1.92<br>flask>=1.1.2<br>ujson>=2.0.3 |
+|          python-whl          | numpy>=1.12, <=1.16.4<br/>wheel>=0.34.0, <0.35.0<br/>setuptools>=44.1.0<br/>opencv-python==4.2.0.32<br/>google>=2.0.3<br/>protobuf>=3.12.2<br/>grpcio-tools>=1.28.1<br/>grpcio>=1.28.1<br/>func-timeout>=4.3.5<br/>pyyaml>=1.3.0<br/>sentencepiece==0.1.92<br>flask>=1.1.2<br>ujson>=2.0.3 |
 
 It is recommended to use Docker for compilation. We have prepared the Paddle Serving compilation environment for you, see [this document](DOCKER_IMAGES.md).
 

doc/COMPILE_CN.md

@@ -23,7 +23,7 @@
 |           libXext            |                            1.3.3                             |
 |            libSM             |                            1.2.2                             |
 |          libXrender          |                            0.9.10                            |
-|          python-whl          | numpy>=1.12, <=1.16.4<br/>google>=2.0.3<br/>protobuf>=3.12.2<br/>grpcio-tools>=1.28.1<br/>grpcio>=1.28.1<br/>func-timeout>=4.3.5<br/>pyyaml>=1.3.0<br/>sentencepiece==0.1.92<br/>flask>=1.1.2<br/>ujson>=2.0.3 |
+|          python-whl          | numpy>=1.12, <=1.16.4<br/>wheel>=0.34.0, <0.35.0<br/>setuptools>=44.1.0<br/>opencv-python==4.2.0.32<br/>google>=2.0.3<br/>protobuf>=3.12.2<br/>grpcio-tools>=1.28.1<br/>grpcio>=1.28.1<br/>func-timeout>=4.3.5<br/>pyyaml>=1.3.0<br/>sentencepiece==0.1.92<br/>flask>=1.1.2<br/>ujson>=2.0.3 |
 
 推荐使用Docker编译，我们已经为您准备好了Paddle Serving编译环境，详见[该文档](DOCKER_IMAGES_CN.md)。
 

python/examples/ocr/README.md

@@ -21,8 +21,13 @@ tar xf test_imgs.tar
 ### Start Service
 
 ```
+#choose one of cpu/gpu commands as following
+#for cpu user
+python -m paddle_serving_server.serve --model ocr_det_model --port 9293
+python ocr_web_server.py cpu
+#for gpu user
 python -m paddle_serving_server_gpu.serve --model ocr_det_model --port 9293 --gpu_id 0
-python ocr_web_server.py
+python ocr_web_server.py gpu
 ```
 
 ### Client Prediction
@@ -33,7 +38,11 @@ If you want a faster web service, please try Web Debugger Service
 
 ## Web Debugger Service
 ```
-python ocr_debugger_server.py
+#choose one of cpu/gpu commands as following
+#for cpu user
+python ocr_debugger_server.py cpu
+#for gpu user
+python ocr_debugger_server.py gpu 
 ```
 
 ## Web Debugger Client Prediction
@@ -54,15 +63,17 @@ Dataset: RCTW 500 sample images
 | Serving web service          | 8.69         | 13.41                      | 109.97           | 2.82               | 87.76            | 4.29               | 3.98               | 78.51            | 3.66               | 4.12                     | 181.02             | 136.49       | 317.51        |
 | Serving Debugger web service |  8.73        | 16.42                      | 115.27           | 2.93               | 20.63            | 3.97               | 4.48               | 13.84            | 3.60               | 6.91                     | 49.45              | 147.33       | 196.78        |
 
-## Appendix: Det or Rec only
+## Appendix: For Users who want to launch Det or Rec only
 if you are going to detect images not recognize it or directly recognize the words from images. We also provide Det and Rec server for you.
 
 ### Det Server 
 
 ```
-python det_web_server.py 
+python det_web_server.py cpu #for cpu user
+python det_web_server.py gpu #for gpu user
 #or
-python det_debugger_server.py
+python det_debugger_server.py cpu #for cpu user
+python det_debugger_server.py gpu #for gpu user
 ```
 
 ### Det Client
@@ -75,9 +86,11 @@ python ocr_web_client.py
 ### Rec Server
 
 ```
-python rec_web_server.py 
+python rec_web_server.py cpu #for cpu user
+python rec_web_server.py gpu #for gpu user
 #or
-python rec_debugger_server.py
+python rec_debugger_server.py cpu #for cpu user
+python rec_debugger_server.py gpu #for gpu user
 ```
 
 ### Rec Client

python/examples/ocr/README_CN.md

@@ -15,19 +15,18 @@ wget --no-check-certificate https://paddle-serving.bj.bcebos.com/ocr/test_imgs.t
 tar xf test_imgs.tar
 ```
 
-### 客户端预测
-
-```
-python ocr_rpc_client.py
-```
-
 ## Web Service服务
 
 ### 启动服务
 
 ```
+#根据CPU/GPU设备选择一种启动方式
+#for cpu user
+python -m paddle_serving_server.serve --model ocr_det_model --port 9293
+python ocr_web_server.py cpu
+#for gpu user
 python -m paddle_serving_server_gpu.serve --model ocr_det_model --port 9293 --gpu_id 0
-python ocr_web_server.py
+python ocr_web_server.py gpu
 ```
 
 ### 启动客户端
@@ -38,7 +37,11 @@ python ocr_web_client.py
 如果用户需要更快的执行速度，请尝试Debugger版Web服务
 ## 启动Debugger版Web服务
 ```
-python ocr_debugger_server.py
+#根据CPU/GPU设备选择一种启动方式
+#for cpu user
+python ocr_debugger_server.py cpu
+#for gpu user
+python ocr_debugger_server.py gpu
 ```
 
 ## 启动客户端
@@ -66,9 +69,11 @@ GPU: Nvidia Tesla V100单卡
 ### 启动检测服务
 
 ```
-python det_web_server.py 
+python det_web_server.py cpu #for cpu user
+python det_web_server.py gpu #for gpu user
 #or
-python det_debugger_server.py
+python det_debugger_server.py cpu #for cpu user
+python det_debugger_server.py gpu #for gpu user
 ```
 
 ### 检测服务客户端
@@ -81,9 +86,11 @@ python ocr_web_client.py
 ### 启动识别服务
 
 ```
-python rec_web_server.py 
+python rec_web_server.py cpu #for cpu user
+python rec_web_server.py gpu #for gpu user
 #or
-python rec_debugger_server.py
+python rec_debugger_server.py cpu #for cpu user
+python rec_debugger_server.py gpu #for gpu user
 ```
 
 ### 识别服务客户端

python/examples/ocr/det_debugger_server.py

@@ -21,7 +21,10 @@ from paddle_serving_client import Client
 from paddle_serving_app.reader import Sequential, ResizeByFactor
 from paddle_serving_app.reader import Div, Normalize, Transpose
 from paddle_serving_app.reader import DBPostProcess, FilterBoxes
-from paddle_serving_server_gpu.web_service import WebService
+if sys.argv[1] == 'gpu':
+    from paddle_serving_server_gpu.web_service import WebService
+elif sys.argv[1] == 'cpu':
+    from paddle_serving_server.web_service import WebService
 import time
 import re
 import base64
@@ -64,8 +67,12 @@ class OCRService(WebService):
 
 ocr_service = OCRService(name="ocr")
 ocr_service.load_model_config("ocr_det_model")
-ocr_service.set_gpus("0")
-ocr_service.prepare_server(workdir="workdir", port=9292, device="gpu", gpuid=0)
+if sys.argv[1] == 'gpu':
+    ocr_service.set_gpus("0")
+    ocr_service.prepare_server(
+        workdir="workdir", port=9292, device="gpu", gpuid=0)
+elif sys.argv[1] == 'cpu':
+    ocr_service.prepare_server(workdir="workdir", port=9292)
 ocr_service.init_det()
 ocr_service.run_debugger_service()
 ocr_service.run_web_service()

python/examples/ocr/det_web_server.py

@@ -21,7 +21,10 @@ from paddle_serving_client import Client
 from paddle_serving_app.reader import Sequential, ResizeByFactor
 from paddle_serving_app.reader import Div, Normalize, Transpose
 from paddle_serving_app.reader import DBPostProcess, FilterBoxes
-from paddle_serving_server_gpu.web_service import WebService
+if sys.argv[1] == 'gpu':
+    from paddle_serving_server_gpu.web_service import WebService
+elif sys.argv[1] == 'cpu':
+    from paddle_serving_server.web_service import WebService
 import time
 import re
 import base64
@@ -65,8 +68,12 @@ class OCRService(WebService):
 
 ocr_service = OCRService(name="ocr")
 ocr_service.load_model_config("ocr_det_model")
-ocr_service.set_gpus("0")
-ocr_service.prepare_server(workdir="workdir", port=9292, device="gpu", gpuid=0)
+if sys.argv[1] == 'gpu':
+    ocr_service.set_gpus("0")
+    ocr_service.prepare_server(
+        workdir="workdir", port=9292, device="gpu", gpuid=0)
+elif sys.argv[1] == 'cpu':
+    ocr_service.prepare_server(workdir="workdir", port=9292, device="cpu")
 ocr_service.init_det()
 ocr_service.run_rpc_service()
 ocr_service.run_web_service()

python/examples/ocr/ocr_debugger_server.py

@@ -22,7 +22,10 @@ from paddle_serving_client import Client
 from paddle_serving_app.reader import Sequential, URL2Image, ResizeByFactor
 from paddle_serving_app.reader import Div, Normalize, Transpose
 from paddle_serving_app.reader import DBPostProcess, FilterBoxes, GetRotateCropImage, SortedBoxes
-from paddle_serving_server_gpu.web_service import WebService
+if sys.argv[1] == 'gpu':
+    from paddle_serving_server_gpu.web_service import WebService
+elif sys.argv[1] == 'cpu':
+    from paddle_serving_server.web_service import WebService
 from paddle_serving_app.local_predict import Debugger
 import time
 import re
@@ -37,8 +40,12 @@ class OCRService(WebService):
                 (2, 0, 1))
         ])
         self.det_client = Debugger()
-        self.det_client.load_model_config(
-            det_model_config, gpu=True, profile=False)
+        if sys.argv[1] == 'gpu':
+            self.det_client.load_model_config(
+                det_model_config, gpu=True, profile=False)
+        elif sys.argv[1] == 'cpu':
+            self.det_client.load_model_config(
+                det_model_config, gpu=False, profile=False)
         self.ocr_reader = OCRReader()
 
     def preprocess(self, feed=[], fetch=[]):
@@ -99,5 +106,8 @@ ocr_service = OCRService(name="ocr")
 ocr_service.load_model_config("ocr_rec_model")
 ocr_service.prepare_server(workdir="workdir", port=9292)
 ocr_service.init_det_debugger(det_model_config="ocr_det_model")
-ocr_service.run_debugger_service(gpu=True)
+if sys.argv[1] == 'gpu':
+    ocr_service.run_debugger_service(gpu=True)
+elif sys.argv[1] == 'cpu':
+    ocr_service.run_debugger_service()
 ocr_service.run_web_service()

python/examples/ocr/ocr_web_server.py

@@ -22,7 +22,10 @@ from paddle_serving_client import Client
 from paddle_serving_app.reader import Sequential, URL2Image, ResizeByFactor
 from paddle_serving_app.reader import Div, Normalize, Transpose
 from paddle_serving_app.reader import DBPostProcess, FilterBoxes, GetRotateCropImage, SortedBoxes
-from paddle_serving_server_gpu.web_service import WebService
+if sys.argv[1] == 'gpu':
+    from paddle_serving_server_gpu.web_service import WebService
+elif sys.argv[1] == 'cpu':
+    from paddle_serving_server.web_service import WebService
 import time
 import re
 import base64
@@ -90,8 +93,12 @@ class OCRService(WebService):
 
 ocr_service = OCRService(name="ocr")
 ocr_service.load_model_config("ocr_rec_model")
-ocr_service.set_gpus("0")
-ocr_service.prepare_server(workdir="workdir", port=9292, device="gpu", gpuid=0)
+if sys.argv[1] == 'gpu':
+    ocr_service.set_gpus("0")
+    ocr_service.prepare_server(
+        workdir="workdir", port=9292, device="gpu", gpuid=0)
+elif sys.argv[1] == 'cpu':
+    ocr_service.prepare_server(workdir="workdir", port=9292)
 ocr_service.init_det_client(
     det_port=9293,
     det_client_config="ocr_det_client/serving_client_conf.prototxt")

python/examples/ocr/rec_web_server.py

@@ -22,7 +22,10 @@ from paddle_serving_client import Client
 from paddle_serving_app.reader import Sequential, URL2Image, ResizeByFactor
 from paddle_serving_app.reader import Div, Normalize, Transpose
 from paddle_serving_app.reader import DBPostProcess, FilterBoxes, GetRotateCropImage, SortedBoxes
-from paddle_serving_server_gpu.web_service import WebService
+if sys.argv[1] == 'gpu':
+    from paddle_serving_server_gpu.web_service import WebService
+elif sys.argv[1] == 'cpu':
+    from paddle_serving_server.web_service import WebService
 import time
 import re
 import base64
@@ -64,8 +67,12 @@ class OCRService(WebService):
 
 ocr_service = OCRService(name="ocr")
 ocr_service.load_model_config("ocr_rec_model")
-ocr_service.set_gpus("0")
 ocr_service.init_rec()
-ocr_service.prepare_server(workdir="workdir", port=9292, device="gpu", gpuid=0)
+if sys.argv[1] == 'gpu':
+    ocr_service.set_gpus("0")
+    ocr_service.prepare_server(
+        workdir="workdir", port=9292, device="gpu", gpuid=0)
+elif sys.argv[1] == 'cpu':
+    ocr_service.prepare_server(workdir="workdir", port=9292, device="cpu")
 ocr_service.run_rpc_service()
 ocr_service.run_web_service()

python/paddle_serving_server/web_service.py

@@ -91,6 +91,8 @@ class WebService(object):
                                           request.json["fetch"])
             if isinstance(feed, dict) and "fetch" in feed:
                 del feed["fetch"]
+            if len(feed) == 0:
+                raise ValueError("empty input")
             fetch_map = self.client.predict(feed=feed, fetch=fetch)
             result = self.postprocess(
                 feed=request.json["feed"], fetch=fetch, fetch_map=fetch_map)
@@ -122,6 +124,32 @@ class WebService(object):
 
         self.app_instance = app_instance
 
+    def run_debugger_service(self):
+        import socket
+        localIP = socket.gethostbyname(socket.gethostname())
+        print("web service address:")
+        print("http://{}:{}/{}/prediction".format(localIP, self.port,
+                                                  self.name))
+        app_instance = Flask(__name__)
+
+        @app_instance.before_first_request
+        def init():
+            self._launch_local_predictor()
+
+        service_name = "/" + self.name + "/prediction"
+
+        @app_instance.route(service_name, methods=["POST"])
+        def run():
+            return self.get_prediction(request)
+
+        self.app_instance = app_instance
+
+    def _launch_local_predictor(self):
+        from paddle_serving_app.local_predict import Debugger
+        self.client = Debugger()
+        self.client.load_model_config(
+            "{}".format(self.model_config), gpu=False, profile=False)
+
     def run_web_service(self):
         self.app_instance.run(host="0.0.0.0",
                               port=self.port,

python/requirements.txt

 numpy>=1.12, <=1.16.4 ; python_version<"3.5"
+wheel>=0.34.0, <0.35.0
+setuptools>=44.1.0
 opencv-python==4.2.0.32
 google>=2.0.3
+opencv-python==4.2.0.32
 protobuf>=3.12.2
 grpcio-tools>=1.28.1
 grpcio>=1.28.1

python/setup.py.app.in

@@ -42,7 +42,7 @@ if '${PACK}' == 'ON':
 
 
 REQUIRED_PACKAGES = [
-    'six >= 1.10.0', 'sentencepiece', 'opencv-python', 'pillow',
+    'six >= 1.10.0', 'sentencepiece', 'opencv-python<=4.2.0.32', 'pillow',
     'shapely', 'pyclipper'
 ]
 

tools/Dockerfile.centos6.cuda9.0-cudnn7.devel

@@ -31,7 +31,6 @@ RUN yum -y install wget && \
     curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
     python get-pip.py && \
     rm get-pip.py && \
-    pip install google protobuf setuptools wheel flask numpy==1.16.4 && \
     wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tgz && \
     tar -zxf Python-3.6.8.tgz && \
     cd Python-3.6.8 && \
@@ -42,7 +41,6 @@ RUN yum -y install wget && \
     echo 'export LD_LIBRARY_PATH=/usr/local/python3.6/lib:$LD_LIBRARY_PATH' >> /root/.bashrc && \
     source /root/.bashrc && \
     cd .. && rm -rf Python-3.6.8* && \
-    pip3 install google protobuf setuptools wheel flask numpy==1.16.4 && \
     yum -y install epel-release && yum -y install patchelf libXext libSM libXrender && \
     yum clean all && \
     echo "export LANG=en_US.utf8" >> /root/.bashrc && \

tools/Dockerfile.centos6.devel

@@ -31,7 +31,6 @@ RUN yum -y install wget && \
     curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \
     python get-pip.py && \
     rm get-pip.py && \
-    pip install google protobuf setuptools wheel flask numpy==1.16.4 && \
     wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tgz && \
     tar -zxf Python-3.6.8.tgz && \
     cd Python-3.6.8 && \
@@ -42,7 +41,6 @@ RUN yum -y install wget && \
     echo 'export LD_LIBRARY_PATH=/usr/local/python3.6/lib:$LD_LIBRARY_PATH' >> /root/.bashrc && \
     source /root/.bashrc && \
     cd .. && rm -rf Python-3.6.8* && \
-    pip3 install google protobuf setuptools wheel flask numpy==1.16.4 && \
     yum -y install epel-release && yum -y install patchelf libXext libSM libXrender && \
     yum clean all && \
     localedef -c -i en_US -f UTF-8 en_US.UTF-8 && \

tools/Dockerfile.ci

@@ -23,7 +23,6 @@ RUN wget https://dl.google.com/go/go1.14.linux-amd64.tar.gz >/dev/null \
 RUN yum -y install python-devel sqlite-devel >/dev/null \
     && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py >/dev/null \
     && python get-pip.py >/dev/null \
-    && pip install google protobuf setuptools wheel flask >/dev/null \
     && rm get-pip.py
 
 RUN wget http://nixos.org/releases/patchelf/patchelf-0.10/patchelf-0.10.tar.bz2 \
@@ -35,8 +34,7 @@ RUN wget http://nixos.org/releases/patchelf/patchelf-0.10/patchelf-0.10.tar.bz2
     && cd .. \
     && rm -rf patchelf-0.10*
 
-RUN yum install -y python3 python3-devel \
-    && pip3 install google protobuf setuptools wheel flask
+RUN yum install -y python3 python3-devel
 
 RUN yum -y update >/dev/null \
     && yum -y install dnf >/dev/null \

tools/Dockerfile.cuda10.0-cudnn7.devel

@@ -23,11 +23,9 @@ RUN wget https://dl.google.com/go/go1.14.linux-amd64.tar.gz >/dev/null \
 RUN yum -y install python-devel sqlite-devel  \
     && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py >/dev/null \
     && python get-pip.py >/dev/null \
-    && pip install google protobuf setuptools wheel flask >/dev/null \
     && rm get-pip.py 
 
 RUN yum install -y python3 python3-devel \
-    && pip3 install google protobuf setuptools wheel flask \
     && yum -y install epel-release && yum -y install patchelf libXext libSM libXrender\
     && yum clean all 
 

tools/Dockerfile.cuda9.0-cudnn7.devel

@@ -22,11 +22,9 @@ RUN wget https://dl.google.com/go/go1.14.linux-amd64.tar.gz >/dev/null \
 RUN yum -y install python-devel sqlite-devel  \
     && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py >/dev/null \
     && python get-pip.py >/dev/null \
-    && pip install google protobuf setuptools wheel flask >/dev/null \
     && rm get-pip.py 
 
 RUN yum install -y python3 python3-devel \
-    && pip3 install google protobuf setuptools wheel flask \
     && yum -y install epel-release && yum -y install patchelf libXext libSM libXrender\
     && yum clean all 
 

tools/Dockerfile.devel

@@ -19,11 +19,9 @@ RUN wget https://dl.google.com/go/go1.14.linux-amd64.tar.gz >/dev/null \
 RUN yum -y install python-devel sqlite-devel  \
     && curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py >/dev/null \
     && python get-pip.py >/dev/null \
-    && pip install google protobuf setuptools wheel flask >/dev/null \
     && rm get-pip.py 
 
 RUN yum install -y python3 python3-devel \
-    && pip3 install google protobuf setuptools wheel flask \
     && yum -y install epel-release && yum -y install patchelf libXext libSM libXrender\
     && yum clean all