diff --git a/deploy/pdserving/README.md b/deploy/pdserving/README.md index 34b525c47214d45819690ed7d161ce3d4dc0c322..cb5e4bfa014a699daa492523918d2ed42fc6cd28 100644 --- a/deploy/pdserving/README.md +++ b/deploy/pdserving/README.md @@ -158,42 +158,43 @@ The recognition model is the same. The predicted performance data will be automatically written into the `PipelineServingLogs/pipeline.tracer` file. - Tested on 200 real pictures, and limited the detection long side to 960. The average QPS on T4 GPU can reach around 13: - - ``` - 2021-05-12 10:03:24,812 ==================== TRACER ====================== - 2021-05-12 10:03:24,904 Op(rec): - 2021-05-12 10:03:24,904 in[51.5634921875 ms] - 2021-05-12 10:03:24,904 prep[215.310859375 ms] - 2021-05-12 10:03:24,904 midp[33.1617109375 ms] - 2021-05-12 10:03:24,905 postp[10.451234375 ms] - 2021-05-12 10:03:24,905 out[9.736765625 ms] - 2021-05-12 10:03:24,905 idle[0.1914292677880819] - 2021-05-12 10:03:24,905 Op(det): - 2021-05-12 10:03:24,905 in[218.63487096774193 ms] - 2021-05-12 10:03:24,906 prep[357.35925 ms] - 2021-05-12 10:03:24,906 midp[31.47598387096774 ms] - 2021-05-12 10:03:24,906 postp[15.274870967741936 ms] - 2021-05-12 10:03:24,906 out[16.245693548387095 ms] - 2021-05-12 10:03:24,906 idle[0.3675805857279226] - 2021-05-12 10:03:24,906 DAGExecutor: - 2021-05-12 10:03:24,906 Query count[128] - 2021-05-12 10:03:24,906 QPS[12.8 q/s] - 2021-05-12 10:03:24,906 Succ[1.0] - 2021-05-12 10:03:24,907 Error req[] - 2021-05-12 10:03:24,907 Latency: - 2021-05-12 10:03:24,907 ave[798.6557734374998 ms] - 2021-05-12 10:03:24,907 .50[867.936 ms] - 2021-05-12 10:03:24,907 .60[914.507 ms] - 2021-05-12 10:03:24,907 .70[961.064 ms] - 2021-05-12 10:03:24,907 .80[1043.264 ms] - 2021-05-12 10:03:24,907 .90[1117.923 ms] - 2021-05-12 10:03:24,907 .95[1207.056 ms] - 2021-05-12 10:03:24,908 .99[1325.008 ms] - 2021-05-12 10:03:24,908 Channel (server worker num[10]): - 2021-05-12 10:03:24,909 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0] - 2021-05-12 10:03:24,909 chl1(In: ['det'], Out: ['rec']) size[1/0] - 2021-05-12 10:03:24,910 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0] + Tested on 200 real pictures, and limited the detection long side to 960. The average QPS on T4 GPU can reach around 23: + + ``` + + 2021-05-13 03:42:36,895 ==================== TRACER ====================== + 2021-05-13 03:42:36,975 Op(rec): + 2021-05-13 03:42:36,976 in[14.472382882882883 ms] + 2021-05-13 03:42:36,976 prep[9.556855855855856 ms] + 2021-05-13 03:42:36,976 midp[59.921905405405404 ms] + 2021-05-13 03:42:36,976 postp[15.345945945945946 ms] + 2021-05-13 03:42:36,976 out[1.9921216216216215 ms] + 2021-05-13 03:42:36,976 idle[0.16254943864471572] + 2021-05-13 03:42:36,976 Op(det): + 2021-05-13 03:42:36,976 in[315.4468035714286 ms] + 2021-05-13 03:42:36,976 prep[69.5980625 ms] + 2021-05-13 03:42:36,976 midp[18.989535714285715 ms] + 2021-05-13 03:42:36,976 postp[18.857803571428573 ms] + 2021-05-13 03:42:36,977 out[3.1337544642857145 ms] + 2021-05-13 03:42:36,977 idle[0.7477961159203756] + 2021-05-13 03:42:36,977 DAGExecutor: + 2021-05-13 03:42:36,977 Query count[224] + 2021-05-13 03:42:36,977 QPS[22.4 q/s] + 2021-05-13 03:42:36,977 Succ[0.9910714285714286] + 2021-05-13 03:42:36,977 Error req[169, 170] + 2021-05-13 03:42:36,977 Latency: + 2021-05-13 03:42:36,977 ave[535.1678348214285 ms] + 2021-05-13 03:42:36,977 .50[172.651 ms] + 2021-05-13 03:42:36,977 .60[187.904 ms] + 2021-05-13 03:42:36,977 .70[245.675 ms] + 2021-05-13 03:42:36,977 .80[526.684 ms] + 2021-05-13 03:42:36,977 .90[854.596 ms] + 2021-05-13 03:42:36,977 .95[1722.728 ms] + 2021-05-13 03:42:36,977 .99[3990.292 ms] + 2021-05-13 03:42:36,978 Channel (server worker num[10]): + 2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0] + 2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0] + 2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0] ``` diff --git a/deploy/pdserving/README_CN.md b/deploy/pdserving/README_CN.md index 4bb3427199ff60968226237bcf67277ae72a1b9d..1e53cb639ce903b7a42840f2e769e63f6894e2ce 100644 --- a/deploy/pdserving/README_CN.md +++ b/deploy/pdserving/README_CN.md @@ -155,42 +155,42 @@ python3 -m paddle_serving_client.convert --dirname ./ch_ppocr_mobile_v2.0_rec_in 预测性能数据会被自动写入 `PipelineServingLogs/pipeline.tracer` 文件中。 - 在200张真实图片上测试，把检测长边限制为960。T4 GPU 上 QPS 均值可达到13左右： - - ``` - 2021-05-12 10:03:24,812 ==================== TRACER ====================== - 2021-05-12 10:03:24,904 Op(rec): - 2021-05-12 10:03:24,904 in[51.5634921875 ms] - 2021-05-12 10:03:24,904 prep[215.310859375 ms] - 2021-05-12 10:03:24,904 midp[33.1617109375 ms] - 2021-05-12 10:03:24,905 postp[10.451234375 ms] - 2021-05-12 10:03:24,905 out[9.736765625 ms] - 2021-05-12 10:03:24,905 idle[0.1914292677880819] - 2021-05-12 10:03:24,905 Op(det): - 2021-05-12 10:03:24,905 in[218.63487096774193 ms] - 2021-05-12 10:03:24,906 prep[357.35925 ms] - 2021-05-12 10:03:24,906 midp[31.47598387096774 ms] - 2021-05-12 10:03:24,906 postp[15.274870967741936 ms] - 2021-05-12 10:03:24,906 out[16.245693548387095 ms] - 2021-05-12 10:03:24,906 idle[0.3675805857279226] - 2021-05-12 10:03:24,906 DAGExecutor: - 2021-05-12 10:03:24,906 Query count[128] - 2021-05-12 10:03:24,906 QPS[12.8 q/s] - 2021-05-12 10:03:24,906 Succ[1.0] - 2021-05-12 10:03:24,907 Error req[] - 2021-05-12 10:03:24,907 Latency: - 2021-05-12 10:03:24,907 ave[798.6557734374998 ms] - 2021-05-12 10:03:24,907 .50[867.936 ms] - 2021-05-12 10:03:24,907 .60[914.507 ms] - 2021-05-12 10:03:24,907 .70[961.064 ms] - 2021-05-12 10:03:24,907 .80[1043.264 ms] - 2021-05-12 10:03:24,907 .90[1117.923 ms] - 2021-05-12 10:03:24,907 .95[1207.056 ms] - 2021-05-12 10:03:24,908 .99[1325.008 ms] - 2021-05-12 10:03:24,908 Channel (server worker num[10]): - 2021-05-12 10:03:24,909 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0] - 2021-05-12 10:03:24,909 chl1(In: ['det'], Out: ['rec']) size[1/0] - 2021-05-12 10:03:24,910 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0] + 在200张真实图片上测试，把检测长边限制为960。T4 GPU 上 QPS 均值可达到23左右： + + ``` + 2021-05-13 03:42:36,895 ==================== TRACER ====================== + 2021-05-13 03:42:36,975 Op(rec): + 2021-05-13 03:42:36,976 in[14.472382882882883 ms] + 2021-05-13 03:42:36,976 prep[9.556855855855856 ms] + 2021-05-13 03:42:36,976 midp[59.921905405405404 ms] + 2021-05-13 03:42:36,976 postp[15.345945945945946 ms] + 2021-05-13 03:42:36,976 out[1.9921216216216215 ms] + 2021-05-13 03:42:36,976 idle[0.16254943864471572] + 2021-05-13 03:42:36,976 Op(det): + 2021-05-13 03:42:36,976 in[315.4468035714286 ms] + 2021-05-13 03:42:36,976 prep[69.5980625 ms] + 2021-05-13 03:42:36,976 midp[18.989535714285715 ms] + 2021-05-13 03:42:36,976 postp[18.857803571428573 ms] + 2021-05-13 03:42:36,977 out[3.1337544642857145 ms] + 2021-05-13 03:42:36,977 idle[0.7477961159203756] + 2021-05-13 03:42:36,977 DAGExecutor: + 2021-05-13 03:42:36,977 Query count[224] + 2021-05-13 03:42:36,977 QPS[22.4 q/s] + 2021-05-13 03:42:36,977 Succ[0.9910714285714286] + 2021-05-13 03:42:36,977 Error req[169, 170] + 2021-05-13 03:42:36,977 Latency: + 2021-05-13 03:42:36,977 ave[535.1678348214285 ms] + 2021-05-13 03:42:36,977 .50[172.651 ms] + 2021-05-13 03:42:36,977 .60[187.904 ms] + 2021-05-13 03:42:36,977 .70[245.675 ms] + 2021-05-13 03:42:36,977 .80[526.684 ms] + 2021-05-13 03:42:36,977 .90[854.596 ms] + 2021-05-13 03:42:36,977 .95[1722.728 ms] + 2021-05-13 03:42:36,977 .99[3990.292 ms] + 2021-05-13 03:42:36,978 Channel (server worker num[10]): + 2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0] + 2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0] + 2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0] ```