Merge pull request #369 from wangjiawei04/jiawei/doc_test

doc test

doc/BERT_10_MINS.md

@@ -8,6 +8,7 @@ The goal of Bert-As-Service is to give a sentence, and the service can represent
 
 Paddle Serving supports various models trained based on Paddle, and saves the serviceable model by specifying the input and output variables of the model. For convenience, we can load a trained bert Chinese model from paddlehub and save a deployable service with two lines of code. The server and client configurations are placed in the `bert_seq20_model` and` bert_seq20_client` folders, respectively.
 
+[//file]:#bert_10.py
 ``` python
 import paddlehub as hub
 model_name = "bert_chinese_L-12_H-768_A-12"
@@ -27,6 +28,7 @@ serving_io.save_model("bert_seq20_model", "bert_seq20_client",
 
 #### Step2: Launch Service
 
+[//file]:#server.sh
 ``` shell
 python -m paddle_serving_server_gpu.serve --model bert_seq20_model --thread 10 --port 9292 --gpu_ids 0
 ```
@@ -43,6 +45,7 @@ Paddle Serving has many built-in corresponding data preprocessing logics. For th
 
 Install paddle_serving_app
 
+[//file]:#pip_app.sh
 ```shell
 pip install paddle_serving_app
 ```
@@ -51,6 +54,7 @@ pip install paddle_serving_app
 
 the script of client side bert_client.py is as follow:
 
+[//file]:#bert_client.py
 ``` python
 import os
 import sys
@@ -71,6 +75,7 @@ for line in sys.stdin:
 
 run
 
+[//file]:#bert_10_cli.sh
 ```shell
 cat data.txt | python bert_client.py
 ```
@@ -82,3 +87,19 @@ read samples from data.txt, print results at the standard output.
 We tested the performance of Bert-As-Service based on Padde Serving based on V100 and compared it with the Bert-As-Service based on Tensorflow. From the perspective of user configuration, we used the same batch size and concurrent number for stress testing. The overall throughput performance data obtained under 4 V100s is as follows.
 
 ![4v100_bert_as_service_benchmark](4v100_bert_as_service_benchmark.png)
+
+<!--
+yum install -y libXext libSM libXrender
+pip install paddlehub paddle_serving_server paddle_serving_client
+sh pip_app.sh
+python bert_10.py
+sh server.sh &
+wget https://paddle-serving.bj.bcebos.com/bert_example/data-c.txt --no-check-certificate
+head -n 500 data-c.txt > data.txt
+cat data.txt | python bert_client.py
+if [[ $? -eq 0 ]]; then
+    echo "test success"
+else
+    echo "test fail"
+fi
+-->

doc/doc_test_list

+BERT_10_MINS.md

tools/doc_test.sh

+#!/usr/bin/env bash
+
+
+function main() {
+   cat Serving/doc/doc_test_list | xargs python Serving/tools/doc_tester_reader.py Serving/doc/ 
+
+}
+
+
+main $@

tools/doc_tester_reader.py

+#   Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import re
+import sys
+
+
+def ReadMarkDown(file):
+    folder = 'test'
+    os.system('rm -rf ' + folder + ' && mkdir -p ' + folder)
+    with open(file, 'r') as f:
+        lines = f.readlines()
+    for i, line in enumerate(lines):
+        if '[//file]:#' in line:
+            filename = line[10:].strip()
+            GetCodeFile(lines, i, os.path.join(folder, filename))
+        if '<!--' in line:
+            filename = 'start.sh'
+            GetTestFile(lines, i, os.path.join(folder, filename))
+
+
+def GetCodeFile(lines, i, filename):
+    if '```' not in lines[i + 1]:
+        raise ValueError(
+            'Syntax Error, code block should be tightly followed by "[//file]:#" '
+        )
+    i += 2
+    code = ''
+    while True:
+        if '```' in lines[i]:
+            break
+        code += lines[i]
+        i += 1
+    with open(filename, 'w+') as f:
+        f.write(code)
+
+
+def GetTestFile(lines, i, filename):
+    i += 1
+    code = ''
+    while True:
+        if '-->' in lines[i]:
+            break
+        code += lines[i]
+        i += 1
+    with open(filename, 'w+') as f:
+        f.write(code)
+
+
+def RunTest():
+    folder = 'test'
+    os.system('cd ' + folder + ' && sh start.sh')
+    os.system('cd .. && rm -rf ' + folder)
+
+
+if __name__ == '__main__':
+    ReadMarkDown(os.path.join(sys.argv[1], sys.argv[2]))
+    RunTest()