diff --git a/README.md b/README.md index a424f9e9b05e636bab2642d9211d42eb31ad1712..669520cb4d9f5061a82718539753385fea67507b 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,7 @@ PaddleClas is a toolset for image classification tasks prepared for the industry and academia. It helps users train better computer vision models and apply them in real scenarios. **Recent update** +- 2020.12.16 Add support for TensorRT when using cpp inference to obain more obvious acceleration. - 2020.12.06 Add `SE_HRNet_W64_C_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 84.75%. - 2020.11.23 Add `GhostNet_x1_3_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 79.38%. - 2020.11.09 Add `InceptionV3` architecture and pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.1%. @@ -15,7 +16,6 @@ PaddleClas is a toolset for image classification tasks prepared for the industry - 2020.10.10 Add cpp inference demo and improve FAQ tutorial. - 2020.09.17 Add `HRNet_W48_C_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 83.62%. Add `ResNet34_vd_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 79.72%. - 2020.09.07 Add `HRNet_W18_C_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 81.16%. -- 2020.07.14 Add `Res2Net200_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 85.13%. Add `Fix_ResNet50_vd_ssld_v2` pretrained model, whose Top-1 Acc on ImageNet-1k dataset reaches 84.00%. - [more](./docs/en/update_history_en.md) @@ -76,7 +76,7 @@ PaddleClas is a toolset for image classification tasks prepared for the industry - [Prediction based on training engine](./docs/en/tutorials/getting_started_en.md) - [Python inference](./docs/en/tutorials/getting_started_en.md) - [C++ inference](./deploy/cpp_infer/readme_en.md) - - [Serving deployment](./docs/en/extension/paddle_serving_en.md) + - [Serving deployment](./deploy/hubserving/readme_en.md) - [Mobile](./deploy/lite/readme_en.md) - [Model Quantization and Compression](docs/en/extension/paddle_quantization_en.md) - Advanced tutorials diff --git a/README_cn.md b/README_cn.md index 89fbeea954cb77b42e78ae83cb8f6e7571c63a90..7902c1c8ab35e651d08555dd3d9bea00ae0fee8d 100644 --- a/README_cn.md +++ b/README_cn.md @@ -8,6 +8,7 @@ **近期更新** +- 2020.12.16 添加对cpp预测的tensorRT支持,预测加速更明显。 - 2020.12.06 添加`SE_HRNet_W64_C_ssld`模型,在ImageNet-1k上Top-1 Acc可达84.75%。 - 2020.11.23 添加`GhostNet_x1_3_ssld `模型,在ImageNet-1k上Top-1 Acc可达79.38%。 - 2020.11.18 添加图像分类[常见问题2020第一季第三期](./docs/zh_CN/faq_series/faq_2020_s1.md) 5个新问题,并且计划以后每周会更新,欢迎大家持续关注。 @@ -17,7 +18,6 @@ - 2020.10.10 添加cpp inference demo,完善`FAQ 30问`教程。 - 2020.09.17 添加 `HRNet_W48_C_ssld `模型,在ImageNet-1k上Top-1 Acc可达83.62%;添加 `ResNet34_vd_ssld `模型,在ImageNet-1k上Top-1 Acc可达79.72%。 - 2020.09.07 添加 `HRNet_W18_C_ssld `模型,在ImageNet-1k上Top-1 Acc可达81.16%;添加 `MobileNetV3_small_x0_35_ssld `模型,在ImageNet-1k上Top-1 Acc可达55.55%。 -- 2020.07.14 添加 `Res2Net200_vd_26w_4s_ssld `模型,在ImageNet-1k上Top-1 Acc可达85.13%;添加 `Fix_ResNet50_vd_ssld_v2 `模型,在ImageNet-1k上Top-1 Acc可达84.0%。 - [more](./docs/zh_CN/update_history.md) @@ -78,7 +78,7 @@ - [基于训练引擎预测推理](./docs/zh_CN/tutorials/getting_started.md) - [基于Python预测引擎预测推理](./docs/zh_CN/tutorials/getting_started.md) - [基于C++预测引擎预测推理](./deploy/cpp_infer/readme.md) - - [服务化部署](./docs/zh_CN/extension/paddle_serving.md) + - [服务化部署](./deploy/hubserving/readme.md) - [端侧部署](./deploy/lite/readme.md) - [模型量化压缩](docs/zh_CN/extension/paddle_quantization.md) - 高阶使用 diff --git a/deploy/cpp_infer/docs/imgs/cpp_infer_result.png b/deploy/cpp_infer/docs/imgs/cpp_infer_result.png index 5873c76aa71dffd2cb504370f3ec50b6929ff156..fe9de1aa43584f1987abd94ad84e64c0d0c78c6a 100644 Binary files a/deploy/cpp_infer/docs/imgs/cpp_infer_result.png and b/deploy/cpp_infer/docs/imgs/cpp_infer_result.png differ diff --git a/deploy/cpp_infer/readme.md b/deploy/cpp_infer/readme.md index db045adff95ab0462b0af64192127403b7331e0e..749414860d635b7f33198260b119986518933735 100644 --- a/deploy/cpp_infer/readme.md +++ b/deploy/cpp_infer/readme.md @@ -164,6 +164,7 @@ OPENCV_DIR=your_opencv_dir LIB_DIR=your_paddle_inference_dir CUDA_LIB_DIR=your_cuda_lib_dir CUDNN_LIB_DIR=your_cudnn_lib_dir +TENSORRT_DIR=your_tensorrt_lib_dir BUILD_DIR=build rm -rf ${BUILD_DIR} @@ -176,6 +177,7 @@ cmake .. \ -DWITH_GPU=OFF \ -DWITH_STATIC_LIB=OFF \ -DWITH_TENSORRT=OFF \ + -DTENSORRT_DIR=${TENSORRT_DIR} \ -DOPENCV_DIR=${OPENCV_DIR} \ -DCUDNN_LIB=${CUDNN_LIB_DIR} \ -DCUDA_LIB=${CUDA_LIB_DIR} \ @@ -193,6 +195,8 @@ make -j * `CUDNN_LIB_DIR`为cudnn库文件地址,在docker中为`/usr/lib/x86_64-linux-gnu/`。 +* `TENSORRT_DIR`是tensorrt库文件地址,在dokcer中为`/usr/local/TensorRT6-cuda10.0-cudnn7/`,TensorRT需要结合GPU使用。 + 在执行上述命令,编译完成之后,会在当前路径下生成`build`文件夹,其中生成一个名为`clas_system`的可执行文件。 @@ -203,6 +207,8 @@ make -j * gpu_mem:显存; * cpu_math_library_num_threads:底层科学计算库所用线程的数量; * use_mkldnn:是否使用MKLDNN加速; + * use_tensorrt: 是否使用tensorRT进行加速; + * use_fp16:是否使用半精度浮点数进行计算,该选项仅在use_tensorrt为true时有效; * cls_model_path:预测模型结构文件路径; * cls_params_path:预测模型参数文件路径; * resize_short_size:预处理时图像缩放大小; diff --git a/deploy/cpp_infer/readme_en.md b/deploy/cpp_infer/readme_en.md index 1d706afd14c6e9c262718cb8e27b8384a8e377a0..fe5abe25624676f8d967b3f63728b43da73c6c99 100644 --- a/deploy/cpp_infer/readme_en.md +++ b/deploy/cpp_infer/readme_en.md @@ -172,6 +172,7 @@ OPENCV_DIR=your_opencv_dir LIB_DIR=your_paddle_inference_dir CUDA_LIB_DIR=your_cuda_lib_dir CUDNN_LIB_DIR=your_cudnn_lib_dir +TENSORRT_DIR=your_tensorrt_lib_dir BUILD_DIR=build rm -rf ${BUILD_DIR} @@ -180,10 +181,11 @@ cd ${BUILD_DIR} cmake .. \ -DPADDLE_LIB=${LIB_DIR} \ -DWITH_MKL=ON \ - -DDEMO_NAME=ocr_system \ + -DDEMO_NAME=clas_system \ -DWITH_GPU=OFF \ -DWITH_STATIC_LIB=OFF \ -DWITH_TENSORRT=OFF \ + -DTENSORRT_DIR=${TENSORRT_DIR} \ -DOPENCV_DIR=${OPENCV_DIR} \ -DCUDNN_LIB=${CUDNN_LIB_DIR} \ -DCUDA_LIB=${CUDA_LIB_DIR} \ @@ -201,11 +203,29 @@ In the above parameters of command: * `CUDNN_LIB_DIR` is the cudnn library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`. +* `TENSORRT_DIR` is the tensorrt library file path,in dokcer it is `/usr/local/TensorRT6-cuda10.0-cudnn7/`,TensorRT is just enabled for GPU. + After the compilation is completed, an executable file named `clas_system` will be generated in the `build` folder. ### Run the demo -* First, please modify the `tools/config.txt` and `tools/run.sh`. Then execute the following command to complete the classification of an image. +* First, please modify the `tools/config.txt` and `tools/run.sh`. + +* Some key words in `tools/config.txt` is as follows. + * use_gpu: Whether to use GPU. + * gpu_id: GPU id. + * gpu_mem:GPU memory. + * cpu_math_library_num_threads:Number of thread for math library acceleration. + * use_mkldnn:Whether to use mkldnn. + * use_tensorrt: Whether to use tensorRT. + * use_fp16:Whether to use Float16 (half precision), it is just enabled when use_tensorrt is set as 1. + * cls_model_path: Model path of inference model. + * cls_params_path: Params path of inference model. + * resize_short_size:Short side length of the image after resize. + * crop_size:Image size after center crop. + + +* Then execute the following command to complete the classification of an image. ```shell sh tools/run.sh diff --git a/docs/en/update_history_en.md b/docs/en/update_history_en.md index 82efc29097d6f92cd312d463bb4fc1891fedd94c..d3e57085953222816a4d4a109cd9a6dd571290fb 100644 --- a/docs/en/update_history_en.md +++ b/docs/en/update_history_en.md @@ -1,14 +1,17 @@ # Release Notes +- 2020.12.16 + * Add support for TensorRT when using cpp inference to obain more obvious acceleration. + - 2020.12.06 * Add `SE_HRNet_W64_C_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 84.75%. - 2020.11.23 * Add `GhostNet_x1_3_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.38%. - + - 2020.11.09 * Add `InceptionV3` architecture and pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 79.1%. - + * 2020.10.20 * Add `Res2Net50_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 83.1%. * Add `Res2Net101_vd_26w_4s_ssld` pretrained model, whose Top-1 Acc on ImageNet1k dataset reaches 83.9%. diff --git a/docs/zh_CN/update_history.md b/docs/zh_CN/update_history.md index bf4de3571766f45076de48e78afafdb6b0cf8f72..5d1d8aa2006c136a1be395a6ea22e5f607dca3f6 100644 --- a/docs/zh_CN/update_history.md +++ b/docs/zh_CN/update_history.md @@ -1,14 +1,17 @@ # 更新日志 +- 2020.12.16 + * 添加对cpp预测的tensorRT支持,预测加速更明显。 + - 2020.12.06 * 添加SE_HRNet_W64_C_ssld模型,在ImageNet上Top-1 Acc可达0.8475。 - + - 2020.11.23 * 添加GhostNet_x1_3_ssld模型,在ImageNet上Top-1 Acc可达0.7938。 - 2020.11.09 * 添加InceptionV3结构和模型,在ImageNet上Top-1 Acc可达0.791。 - + - 2020.10.20 * 添加Res2Net50_vd_26w_4s_ssld模型,在ImageNet上Top-1 Acc可达0.831;添加Res2Net101_vd_26w_4s_ssld模型,在ImageNet上Top-1 Acc可达0.839。