提交 b953f871 编写于 作者: C Channingss

merge paddle/develop

...@@ -3,4 +3,4 @@ name: 4. PaddleX GUI使用问题 ...@@ -3,4 +3,4 @@ name: 4. PaddleX GUI使用问题
about: Paddle GUI客户端使用问题 about: Paddle GUI客户端使用问题
--- ---
PaddleX GUI: https://www.paddlepaddle.org.cn/paddle/paddleX PaddleX GUI: https://www.paddlepaddle.org.cn/paddle/paddleX (请在ISSUE内容中保留此行内容)
<p align="center"> <p align="center">
<img src="./docs/gui/images/paddlex.png" width="360" height ="55" alt="PaddleX" align="middle" /> <img src="./docs/gui/images/paddlex.png" width="360" height ="55" alt="PaddleX" align="middle" />
</p> </p>
<p align= "center"> PaddleX -- 飞桨全流程开发套件,以低代码的形式支持开发者快速实现产业实际项目落地 </p>
[![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE) [![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE)
[![Version](https://img.shields.io/github/release/PaddlePaddle/PaddleX.svg)](https://github.com/PaddlePaddle/PaddleX/releases) [![Version](https://img.shields.io/github/release/PaddlePaddle/PaddleX.svg)](https://github.com/PaddlePaddle/PaddleX/releases)
![python version](https://img.shields.io/badge/python-3.6+-orange.svg) ![python version](https://img.shields.io/badge/python-3.6+-orange.svg)
![support os](https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-yellow.svg) ![support os](https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-yellow.svg)
![QQGroup](https://img.shields.io/badge/QQ_Group-1045148026-52B6EF?style=social&logo=tencent-qq&logoColor=000&logoWidth=20) ![QQGroup](https://img.shields.io/badge/QQ_Group-1045148026-52B6EF?style=social&logo=tencent-qq&logoColor=000&logoWidth=20)
**PaddleX--飞桨全功能开发套件**,集成了飞桨视觉套件(PaddleClas、PaddleDetection、PaddleSeg)、模型压缩工具PaddleSlim、可视化分析工具VisualDL、轻量化推理引擎Paddle Lite 等核心模块的能力,同时融合飞桨团队丰富的实际经验及技术积累,将深度学习开发全流程,从数据准备、模型训练与优化到多端部署实现了端到端打通,为开发者提供飞桨全流程开发的最佳实践。 集成飞桨智能视觉领域**图像分类****目标检测****语义分割****实例分割**任务能力,将深度学习开发全流程从**数据准备****模型训练与优化****多端部署**端到端打通,并提供**统一任务API接口****图形化开发界面Demo**。开发者无需分别安装不同套件,以**低代码**的形式即可快速完成飞桨全流程开发。
**PaddleX 提供了最简化的API设计,并官方实现GUI供大家下载使用**,最大程度降低开发者使用门槛。开发者既可以应用**PaddleX GUI**快速体验深度学习模型开发的全流程,也可以直接使用 **PaddleX API** 更灵活地进行开发。
更进一步的,如果用户需要根据自己场景及需求,定制化地对PaddleX 进行改造或集成,PaddleX 也提供很好的支持。
## PaddleX 三大特点 **PaddleX** 经过**质检****安防****巡检****遥感****零售****医疗**等十多个行业实际应用场景验证,沉淀产业实际经验,**并提供丰富的案例实践教程**,全程助力开发者产业实践落地。
### 全流程打通
- **数据准备**:兼容ImageNet、VOC、COCO等常用数据协议, 同时与Labelme、精灵标注助手、[EasyData智能数据服务平台](https://ai.baidu.com/easydata/)等无缝衔接,全方位助力开发者更快完成数据准备工作。
- **数据预处理及增强**:提供极简的图像预处理和增强方法--Transforms,适配imgaug图像增强库,支持上百种数据增强策略,是开发者快速缓解小样本数据训练的问题。
- **模型训练**:集成[PaddleClas](https://github.com/PaddlePaddle/PaddleClas), [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), [PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)视觉开发套件,提供大量精选的、经过产业实践的高质量预训练模型,使开发者更快实现工业级模型效果。
- **模型调优**:内置模型可解释性模块、[VisualDL](https://github.com/PaddlePaddle/VisualDL)可视化分析工具。使开发者可以更直观的理解模型的特征提取区域、训练过程参数变化,从而快速优化模型。
- **多端安全部署**:内置[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)模型压缩工具和**模型加密部署模块**,与飞桨原生预测库Paddle Inference及高性能端侧推理引擎[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) 无缝打通,使开发者快速实现模型的多端、高性能、安全部署。
### 融合产业实践
- **产业验证**:经过**质检****安防****巡检****遥感****零售****医疗**等十多个行业实际应用场景验证,适配行业数据格式及部署环境要求。
- **经验沉淀**:沉淀产业实践实际经验,**提供丰富的案例实践教程**,加速开发者产业落地。
- **产业开发者共建**:吸收实际产业开发者贡献代码,源于产业,回馈产业。
## 易用易集成
- **易用**:统一的全流程API,5步即可完成模型训练,10行代码实现Python/C++高性能部署。
- **易集成**:支持开发者自主改造、集成,开发出适用于自己产业需求的产品。并官方提供基于 PaddleX API 开发的跨平台可视化工具-- **PaddleX GUI**,使开发者快速体验飞桨深度学习开发全流程,并启发用户进行定制化开发。
## 安装 ## 安装
**PaddleX提供两种开发模式,满足用户的不同需求:** **PaddleX提供两种开发模式,满足用户的不同需求:**
1. **Python开发模式:** 通过简洁易懂的Python API,在兼顾功能全面性、开发灵活性、集成方便性的基础上,给开发者最流畅的深度学习开发体验。<br> 1. **Python开发模式:**
通过简洁易懂的Python API,在兼顾功能全面性、开发灵活性、集成方便性的基础上,给开发者最流畅的深度学习开发体验。<br>
**前置依赖** **前置依赖**
> - paddlepaddle >= 1.8.0 > - paddlepaddle >= 1.8.0
> - python >= 3.5 > - python >= 3.6
> - cython > - cython
> - pycocotools > - pycocotools
...@@ -59,12 +40,29 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple ...@@ -59,12 +40,29 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple
详细安装方法请参考[PaddleX安装](https://paddlex.readthedocs.io/zh_CN/develop/install.html) 详细安装方法请参考[PaddleX安装](https://paddlex.readthedocs.io/zh_CN/develop/install.html)
2. **Padlde GUI模式:** 无代码开发的可视化客户端,应用Paddle API实现,使开发者快速进行产业项目验证,并为用户开发自有深度学习软件/应用提供参照。 2. **Padlde GUI模式:**
无代码开发的可视化客户端,应用Paddle API实现,使开发者快速进行产业项目验证,并为用户开发自有深度学习软件/应用提供参照。
- 前往[PaddleX官网](https://www.paddlepaddle.org.cn/paddle/paddlex),申请下载Paddle X GUI一键绿色安装包。 - 前往[PaddleX官网](https://www.paddlepaddle.org.cn/paddle/paddlex),申请下载Paddle X GUI一键绿色安装包。
- 前往[PaddleX GUI使用教程](./docs/gui/how_to_use.md)了解PaddleX GUI使用详情。 - 前往[PaddleX GUI使用教程](./docs/gui/how_to_use.md)了解PaddleX GUI使用详情。
## 产品模块说明
- **数据准备**:兼容ImageNet、VOC、COCO等常用数据协议,同时与Labelme、精灵标注助手、[EasyData智能数据服务平台](https://ai.baidu.com/easydata/)等无缝衔接,全方位助力开发者更快完成数据准备工作。
- **数据预处理及增强**:提供极简的图像预处理和增强方法--Transforms,适配imgaug图像增强库,支持**上百种数据增强策略**,是开发者快速缓解小样本数据训练的问题。
- **模型训练**:集成[PaddleClas](https://github.com/PaddlePaddle/PaddleClas), [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), [PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)视觉开发套件,提供大量精选的、经过产业实践的高质量预训练模型,使开发者更快实现工业级模型效果。
- **模型调优**:内置模型可解释性模块、[VisualDL](https://github.com/PaddlePaddle/VisualDL)可视化分析工具。使开发者可以更直观的理解模型的特征提取区域、训练过程参数变化,从而快速优化模型。
- **多端安全部署**:内置[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)模型压缩工具和**模型加密部署模块**,与飞桨原生预测库Paddle Inference及高性能端侧推理引擎[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) 无缝打通,使开发者快速实现模型的多端、高性能、安全部署。
## 完整使用文档及API说明 ## 完整使用文档及API说明
...@@ -74,7 +72,7 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple ...@@ -74,7 +72,7 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple
- [PaddleX模型训练教程集合](https://paddlex.readthedocs.io/zh_CN/develop/train/index.html) - [PaddleX模型训练教程集合](https://paddlex.readthedocs.io/zh_CN/develop/train/index.html)
- [PaddleX API接口说明](https://paddlex.readthedocs.io/zh_CN/develop/apis/index.html) - [PaddleX API接口说明](https://paddlex.readthedocs.io/zh_CN/develop/apis/index.html)
## 在线项目示例 ### 在线项目示例
为了使开发者更快掌握PaddleX API,我们创建了一系列完整的示例教程,您可通过AIStudio一站式开发平台,快速在线运行PaddleX的项目。 为了使开发者更快掌握PaddleX API,我们创建了一系列完整的示例教程,您可通过AIStudio一站式开发平台,快速在线运行PaddleX的项目。
...@@ -83,15 +81,36 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple ...@@ -83,15 +81,36 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple
- [PaddleX快速上手——Faster-RCNN AI识虫](https://aistudio.baidu.com/aistudio/projectdetail/439888) - [PaddleX快速上手——Faster-RCNN AI识虫](https://aistudio.baidu.com/aistudio/projectdetail/439888)
- [PaddleX快速上手——DeepLabv3+ 视盘分割](https://aistudio.baidu.com/aistudio/projectdetail/440197) - [PaddleX快速上手——DeepLabv3+ 视盘分割](https://aistudio.baidu.com/aistudio/projectdetail/440197)
## 交流与反馈
- 项目官网: https://www.paddlepaddle.org.cn/paddle/paddlex
- PaddleX用户交流群: 1045148026 (手机QQ扫描如下二维码快速加入) ## 全流程产业应用案例
<img src="./docs/gui/images/QR.jpg" width="250" height="300" alt="QQGroup" align="center" />
(continue to be updated)
* 工业巡检:
* [工业表计读数](https://paddlex.readthedocs.io/zh_CN/develop/examples/meter_reader.html)
* 工业质检:
* 电池隔膜缺陷检测(Coming Soon)
* [人像分割](https://paddlex.readthedocs.io/zh_CN/develop/examples/human_segmentation.html)
## [FAQ](./docs/gui/faq.md) ## [FAQ](./docs/gui/faq.md)
## 交流与反馈
- 项目官网:https://www.paddlepaddle.org.cn/paddle/paddlex
- PaddleX用户交流群:957286141 (手机QQ扫描如下二维码快速加入)
![](./docs/gui/images/QR2.jpg)
## 更新日志 ## 更新日志
> [历史版本及更新内容](https://paddlex.readthedocs.io/zh_CN/develop/change_log.html) > [历史版本及更新内容](https://paddlex.readthedocs.io/zh_CN/develop/change_log.html)
- 2020.07.13 v1.1.0 - 2020.07.13 v1.1.0
...@@ -99,6 +118,8 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple ...@@ -99,6 +118,8 @@ pip install paddlex -i https://mirror.baidu.com/pypi/simple
- 2020.05.20 v1.0.0 - 2020.05.20 v1.0.0
- 2020.05.17 v0.1.8 - 2020.05.17 v0.1.8
## 贡献代码 ## 贡献代码
我们非常欢迎您为PaddleX贡献代码或者提供使用建议。如果您可以修复某个issue或者增加一个新功能,欢迎给我们提交Pull Requests. 我们非常欢迎您为PaddleX贡献代码或者提供使用建议。如果您可以修复某个issue或者增加一个新功能,欢迎给我们提交Pull Requests
...@@ -17,7 +17,6 @@ SET(OPENCV_DIR "" CACHE PATH "Location of libraries") ...@@ -17,7 +17,6 @@ SET(OPENCV_DIR "" CACHE PATH "Location of libraries")
SET(ENCRYPTION_DIR"" CACHE PATH "Location of libraries") SET(ENCRYPTION_DIR"" CACHE PATH "Location of libraries")
SET(CUDA_LIB "" CACHE PATH "Location of libraries") SET(CUDA_LIB "" CACHE PATH "Location of libraries")
if (NOT WIN32) if (NOT WIN32)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib) set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib) set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
...@@ -51,7 +50,9 @@ endmacro() ...@@ -51,7 +50,9 @@ endmacro()
if (WITH_ENCRYPTION) if (WITH_ENCRYPTION)
add_definitions( -DWITH_ENCRYPTION=${WITH_ENCRYPTION}) if (NOT (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64"))
add_definitions( -DWITH_ENCRYPTION=${WITH_ENCRYPTION})
endif()
endif() endif()
if (WITH_MKL) if (WITH_MKL)
...@@ -62,8 +63,10 @@ if (NOT DEFINED PADDLE_DIR OR ${PADDLE_DIR} STREQUAL "") ...@@ -62,8 +63,10 @@ if (NOT DEFINED PADDLE_DIR OR ${PADDLE_DIR} STREQUAL "")
message(FATAL_ERROR "please set PADDLE_DIR with -DPADDLE_DIR=/path/paddle_influence_dir") message(FATAL_ERROR "please set PADDLE_DIR with -DPADDLE_DIR=/path/paddle_influence_dir")
endif() endif()
if (NOT DEFINED OPENCV_DIR OR ${OPENCV_DIR} STREQUAL "") if (NOT (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64"))
if (NOT DEFINED OPENCV_DIR OR ${OPENCV_DIR} STREQUAL "")
message(FATAL_ERROR "please set OPENCV_DIR with -DOPENCV_DIR=/path/opencv") message(FATAL_ERROR "please set OPENCV_DIR with -DOPENCV_DIR=/path/opencv")
endif()
endif() endif()
include_directories("${CMAKE_SOURCE_DIR}/") include_directories("${CMAKE_SOURCE_DIR}/")
...@@ -111,10 +114,17 @@ if (WIN32) ...@@ -111,10 +114,17 @@ if (WIN32)
find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/build/ NO_DEFAULT_PATH) find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/build/ NO_DEFAULT_PATH)
unset(OpenCV_DIR CACHE) unset(OpenCV_DIR CACHE)
else () else ()
find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/share/OpenCV NO_DEFAULT_PATH) if (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64") # x86_64 aarch64
set(OpenCV_INCLUDE_DIRS "/usr/include/opencv4")
file(GLOB OpenCV_LIBS /usr/lib/aarch64-linux-gnu/libopencv_*${CMAKE_SHARED_LIBRARY_SUFFIX})
message("OpenCV libs: ${OpenCV_LIBS}")
else()
find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/share/OpenCV NO_DEFAULT_PATH)
endif()
include_directories("${PADDLE_DIR}/paddle/include") include_directories("${PADDLE_DIR}/paddle/include")
link_directories("${PADDLE_DIR}/paddle/lib") link_directories("${PADDLE_DIR}/paddle/lib")
endif () endif ()
include_directories(${OpenCV_INCLUDE_DIRS}) include_directories(${OpenCV_INCLUDE_DIRS})
if (WIN32) if (WIN32)
...@@ -260,9 +270,11 @@ endif() ...@@ -260,9 +270,11 @@ endif()
if(WITH_ENCRYPTION) if(WITH_ENCRYPTION)
if(NOT WIN32) if(NOT WIN32)
if (NOT (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64"))
include_directories("${ENCRYPTION_DIR}/include") include_directories("${ENCRYPTION_DIR}/include")
link_directories("${ENCRYPTION_DIR}/lib") link_directories("${ENCRYPTION_DIR}/lib")
set(DEPS ${DEPS} ${ENCRYPTION_DIR}/lib/libpmodel-decrypt${CMAKE_SHARED_LIBRARY_SUFFIX}) set(DEPS ${DEPS} ${ENCRYPTION_DIR}/lib/libpmodel-decrypt${CMAKE_SHARED_LIBRARY_SUFFIX})
endif()
else() else()
include_directories("${ENCRYPTION_DIR}/include") include_directories("${ENCRYPTION_DIR}/include")
link_directories("${ENCRYPTION_DIR}/lib") link_directories("${ENCRYPTION_DIR}/lib")
...@@ -276,6 +288,7 @@ if (NOT WIN32) ...@@ -276,6 +288,7 @@ if (NOT WIN32)
endif() endif()
set(DEPS ${DEPS} ${OpenCV_LIBS}) set(DEPS ${DEPS} ${OpenCV_LIBS})
add_library(paddlex_inference SHARED src/visualize src/transforms.cpp src/paddlex.cpp) add_library(paddlex_inference SHARED src/visualize src/transforms.cpp src/paddlex.cpp)
ADD_DEPENDENCIES(paddlex_inference ext-yaml-cpp) ADD_DEPENDENCIES(paddlex_inference ext-yaml-cpp)
target_link_libraries(paddlex_inference ${DEPS}) target_link_libraries(paddlex_inference ${DEPS})
...@@ -292,6 +305,19 @@ add_executable(segmenter demo/segmenter.cpp src/transforms.cpp src/paddlex.cpp s ...@@ -292,6 +305,19 @@ add_executable(segmenter demo/segmenter.cpp src/transforms.cpp src/paddlex.cpp s
ADD_DEPENDENCIES(segmenter ext-yaml-cpp) ADD_DEPENDENCIES(segmenter ext-yaml-cpp)
target_link_libraries(segmenter ${DEPS}) target_link_libraries(segmenter ${DEPS})
add_executable(video_classifier demo/video_classifier.cpp src/transforms.cpp src/paddlex.cpp src/visualize.cpp)
ADD_DEPENDENCIES(video_classifier ext-yaml-cpp)
target_link_libraries(video_classifier ${DEPS})
add_executable(video_detector demo/video_detector.cpp src/transforms.cpp src/paddlex.cpp src/visualize.cpp)
ADD_DEPENDENCIES(video_detector ext-yaml-cpp)
target_link_libraries(video_detector ${DEPS})
add_executable(video_segmenter demo/video_segmenter.cpp src/transforms.cpp src/paddlex.cpp src/visualize.cpp)
ADD_DEPENDENCIES(video_segmenter ext-yaml-cpp)
target_link_libraries(video_segmenter ${DEPS})
if (WIN32 AND WITH_MKL) if (WIN32 AND WITH_MKL)
add_custom_command(TARGET classifier POST_BUILD add_custom_command(TARGET classifier POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./paddlex_inference/Release/mklml.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./paddlex_inference/Release/mklml.dll
...@@ -313,7 +339,27 @@ if (WIN32 AND WITH_MKL) ...@@ -313,7 +339,27 @@ if (WIN32 AND WITH_MKL)
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./paddlex_inference/Release/mkldnn.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./paddlex_inference/Release/mkldnn.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./release/mkldnn.dll )
add_custom_command(TARGET video_classifier POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./paddlex_inference/Release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./paddlex_inference/Release/libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./paddlex_inference/Release/mkldnn.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll
)
add_custom_command(TARGET video_detector POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./paddlex_inference/Release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./paddlex_inference/Release/libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./paddlex_inference/Release/mkldnn.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll
)
add_custom_command(TARGET video_segmenter POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./paddlex_inference/Release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./paddlex_inference/Release/libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./paddlex_inference/Release/mkldnn.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll
) )
# for encryption # for encryption
if (EXISTS "${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll") if (EXISTS "${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll")
...@@ -329,6 +375,18 @@ if (WIN32 AND WITH_MKL) ...@@ -329,6 +375,18 @@ if (WIN32 AND WITH_MKL)
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll
) )
add_custom_command(TARGET video_classifier POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll
)
add_custom_command(TARGET video_detector POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll
)
add_custom_command(TARGET video_segmenter POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll
)
endif() endif()
endif() endif()
......
...@@ -37,7 +37,6 @@ DEFINE_int32(batch_size, 1, "Batch size of infering"); ...@@ -37,7 +37,6 @@ DEFINE_int32(batch_size, 1, "Batch size of infering");
DEFINE_int32(thread_num, DEFINE_int32(thread_num,
omp_get_num_procs(), omp_get_num_procs(),
"Number of preprocessing threads"); "Number of preprocessing threads");
DEFINE_bool(use_ir_optim, true, "use ir optimization");
int main(int argc, char** argv) { int main(int argc, char** argv) {
// Parsing command-line // Parsing command-line
...@@ -52,16 +51,15 @@ int main(int argc, char** argv) { ...@@ -52,16 +51,15 @@ int main(int argc, char** argv) {
return -1; return -1;
} }
// 加载模型 // Load model
PaddleX::Model model; PaddleX::Model model;
model.Init(FLAGS_model_dir, model.Init(FLAGS_model_dir,
FLAGS_use_gpu, FLAGS_use_gpu,
FLAGS_use_trt, FLAGS_use_trt,
FLAGS_gpu_id, FLAGS_gpu_id,
FLAGS_key, FLAGS_key);
FLAGS_use_ir_optim);
// 进行预测 // Predict
int imgs = 1; int imgs = 1;
if (FLAGS_image_list != "") { if (FLAGS_image_list != "") {
std::ifstream inf(FLAGS_image_list); std::ifstream inf(FLAGS_image_list);
...@@ -69,7 +67,7 @@ int main(int argc, char** argv) { ...@@ -69,7 +67,7 @@ int main(int argc, char** argv) {
std::cerr << "Fail to open file " << FLAGS_image_list << std::endl; std::cerr << "Fail to open file " << FLAGS_image_list << std::endl;
return -1; return -1;
} }
// 多batch预测 // Mini-batch predict
std::string image_path; std::string image_path;
std::vector<std::string> image_paths; std::vector<std::string> image_paths;
while (getline(inf, image_path)) { while (getline(inf, image_path)) {
...@@ -77,7 +75,7 @@ int main(int argc, char** argv) { ...@@ -77,7 +75,7 @@ int main(int argc, char** argv) {
} }
imgs = image_paths.size(); imgs = image_paths.size();
for (int i = 0; i < image_paths.size(); i += FLAGS_batch_size) { for (int i = 0; i < image_paths.size(); i += FLAGS_batch_size) {
// 读图像 // Read image
int im_vec_size = int im_vec_size =
std::min(static_cast<int>(image_paths.size()), i + FLAGS_batch_size); std::min(static_cast<int>(image_paths.size()), i + FLAGS_batch_size);
std::vector<cv::Mat> im_vec(im_vec_size - i); std::vector<cv::Mat> im_vec(im_vec_size - i);
......
...@@ -43,10 +43,9 @@ DEFINE_double(threshold, ...@@ -43,10 +43,9 @@ DEFINE_double(threshold,
DEFINE_int32(thread_num, DEFINE_int32(thread_num,
omp_get_num_procs(), omp_get_num_procs(),
"Number of preprocessing threads"); "Number of preprocessing threads");
DEFINE_bool(use_ir_optim, true, "use ir optimization");
int main(int argc, char** argv) { int main(int argc, char** argv) {
// 解析命令行参数 // Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true); google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") { if (FLAGS_model_dir == "") {
...@@ -57,17 +56,16 @@ int main(int argc, char** argv) { ...@@ -57,17 +56,16 @@ int main(int argc, char** argv) {
std::cerr << "--image or --image_list need to be defined" << std::endl; std::cerr << "--image or --image_list need to be defined" << std::endl;
return -1; return -1;
} }
// 加载模型 // Load model
PaddleX::Model model; PaddleX::Model model;
model.Init(FLAGS_model_dir, model.Init(FLAGS_model_dir,
FLAGS_use_gpu, FLAGS_use_gpu,
FLAGS_use_trt, FLAGS_use_trt,
FLAGS_gpu_id, FLAGS_gpu_id,
FLAGS_key, FLAGS_key);
FLAGS_use_ir_optim);
int imgs = 1; int imgs = 1;
std::string save_dir = "output"; std::string save_dir = "output";
// 进行预测 // Predict
if (FLAGS_image_list != "") { if (FLAGS_image_list != "") {
std::ifstream inf(FLAGS_image_list); std::ifstream inf(FLAGS_image_list);
if (!inf) { if (!inf) {
...@@ -92,7 +90,7 @@ int main(int argc, char** argv) { ...@@ -92,7 +90,7 @@ int main(int argc, char** argv) {
im_vec[j - i] = std::move(cv::imread(image_paths[j], 1)); im_vec[j - i] = std::move(cv::imread(image_paths[j], 1));
} }
model.predict(im_vec, &results, thread_num); model.predict(im_vec, &results, thread_num);
// 输出结果目标框 // Output predicted bounding boxes
for (int j = 0; j < im_vec_size - i; ++j) { for (int j = 0; j < im_vec_size - i; ++j) {
for (int k = 0; k < results[j].boxes.size(); ++k) { for (int k = 0; k < results[j].boxes.size(); ++k) {
std::cout << "image file: " << image_paths[i + j] << ", "; std::cout << "image file: " << image_paths[i + j] << ", ";
...@@ -106,7 +104,7 @@ int main(int argc, char** argv) { ...@@ -106,7 +104,7 @@ int main(int argc, char** argv) {
<< results[j].boxes[k].coordinate[3] << ")" << std::endl; << results[j].boxes[k].coordinate[3] << ")" << std::endl;
} }
} }
// 可视化 // Visualize results
for (int j = 0; j < im_vec_size - i; ++j) { for (int j = 0; j < im_vec_size - i; ++j) {
cv::Mat vis_img = PaddleX::Visualize( cv::Mat vis_img = PaddleX::Visualize(
im_vec[j], results[j], model.labels, FLAGS_threshold); im_vec[j], results[j], model.labels, FLAGS_threshold);
...@@ -120,7 +118,7 @@ int main(int argc, char** argv) { ...@@ -120,7 +118,7 @@ int main(int argc, char** argv) {
PaddleX::DetResult result; PaddleX::DetResult result;
cv::Mat im = cv::imread(FLAGS_image, 1); cv::Mat im = cv::imread(FLAGS_image, 1);
model.predict(im, &result); model.predict(im, &result);
// 输出结果目标框 // Output predicted bounding boxes
for (int i = 0; i < result.boxes.size(); ++i) { for (int i = 0; i < result.boxes.size(); ++i) {
std::cout << "image file: " << FLAGS_image << std::endl; std::cout << "image file: " << FLAGS_image << std::endl;
std::cout << ", predict label: " << result.boxes[i].category std::cout << ", predict label: " << result.boxes[i].category
...@@ -132,7 +130,7 @@ int main(int argc, char** argv) { ...@@ -132,7 +130,7 @@ int main(int argc, char** argv) {
<< result.boxes[i].coordinate[3] << ")" << std::endl; << result.boxes[i].coordinate[3] << ")" << std::endl;
} }
// 可视化 // Visualize results
cv::Mat vis_img = cv::Mat vis_img =
PaddleX::Visualize(im, result, model.labels, FLAGS_threshold); PaddleX::Visualize(im, result, model.labels, FLAGS_threshold);
std::string save_path = std::string save_path =
......
...@@ -39,10 +39,9 @@ DEFINE_int32(batch_size, 1, "Batch size of infering"); ...@@ -39,10 +39,9 @@ DEFINE_int32(batch_size, 1, "Batch size of infering");
DEFINE_int32(thread_num, DEFINE_int32(thread_num,
omp_get_num_procs(), omp_get_num_procs(),
"Number of preprocessing threads"); "Number of preprocessing threads");
DEFINE_bool(use_ir_optim, false, "use ir optimization");
int main(int argc, char** argv) { int main(int argc, char** argv) {
// 解析命令行参数 // Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true); google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") { if (FLAGS_model_dir == "") {
...@@ -54,16 +53,15 @@ int main(int argc, char** argv) { ...@@ -54,16 +53,15 @@ int main(int argc, char** argv) {
return -1; return -1;
} }
// 加载模型 // Load model
PaddleX::Model model; PaddleX::Model model;
model.Init(FLAGS_model_dir, model.Init(FLAGS_model_dir,
FLAGS_use_gpu, FLAGS_use_gpu,
FLAGS_use_trt, FLAGS_use_trt,
FLAGS_gpu_id, FLAGS_gpu_id,
FLAGS_key, FLAGS_key);
FLAGS_use_ir_optim);
int imgs = 1; int imgs = 1;
// 进行预测 // Predict
if (FLAGS_image_list != "") { if (FLAGS_image_list != "") {
std::ifstream inf(FLAGS_image_list); std::ifstream inf(FLAGS_image_list);
if (!inf) { if (!inf) {
...@@ -88,7 +86,7 @@ int main(int argc, char** argv) { ...@@ -88,7 +86,7 @@ int main(int argc, char** argv) {
im_vec[j - i] = std::move(cv::imread(image_paths[j], 1)); im_vec[j - i] = std::move(cv::imread(image_paths[j], 1));
} }
model.predict(im_vec, &results, thread_num); model.predict(im_vec, &results, thread_num);
// 可视化 // Visualize results
for (int j = 0; j < im_vec_size - i; ++j) { for (int j = 0; j < im_vec_size - i; ++j) {
cv::Mat vis_img = cv::Mat vis_img =
PaddleX::Visualize(im_vec[j], results[j], model.labels); PaddleX::Visualize(im_vec[j], results[j], model.labels);
...@@ -102,7 +100,7 @@ int main(int argc, char** argv) { ...@@ -102,7 +100,7 @@ int main(int argc, char** argv) {
PaddleX::SegResult result; PaddleX::SegResult result;
cv::Mat im = cv::imread(FLAGS_image, 1); cv::Mat im = cv::imread(FLAGS_image, 1);
model.predict(im, &result); model.predict(im, &result);
// 可视化 // Visualize results
cv::Mat vis_img = PaddleX::Visualize(im, result, model.labels); cv::Mat vis_img = PaddleX::Visualize(im, result, model.labels);
std::string save_path = std::string save_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_image); PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_image);
......
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <glog/logging.h>
#include <omp.h>
#include <algorithm>
#include <chrono> // NOLINT
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <utility>
#include "include/paddlex/paddlex.h"
#include "include/paddlex/visualize.h"
#if defined(__arm__) || defined(__aarch64__)
#include <opencv2/videoio/legacy/constants_c.h>
#endif
using namespace std::chrono; // NOLINT
DEFINE_string(model_dir, "", "Path of inference model");
DEFINE_bool(use_gpu, false, "Infering with GPU or CPU");
DEFINE_bool(use_trt, false, "Infering with TensorRT");
DEFINE_int32(gpu_id, 0, "GPU card id");
DEFINE_string(key, "", "key of encryption");
DEFINE_bool(use_camera, false, "Infering with Camera");
DEFINE_int32(camera_id, 0, "Camera id");
DEFINE_string(video_path, "", "Path of input video");
DEFINE_bool(show_result, false, "show the result of each frame with a window");
DEFINE_bool(save_result, true, "save the result of each frame to a video");
DEFINE_string(save_dir, "output", "Path to save visualized image");
int main(int argc, char** argv) {
// Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") {
std::cerr << "--model_dir need to be defined" << std::endl;
return -1;
}
if (FLAGS_video_path == "" & FLAGS_use_camera == false) {
std::cerr << "--video_path or --use_camera need to be defined" << std::endl;
return -1;
}
// Load model
PaddleX::Model model;
model.Init(FLAGS_model_dir,
FLAGS_use_gpu,
FLAGS_use_trt,
FLAGS_gpu_id,
FLAGS_key);
// Open video
cv::VideoCapture capture;
if (FLAGS_use_camera) {
capture.open(FLAGS_camera_id);
if (!capture.isOpened()) {
std::cout << "Can not open the camera "
<< FLAGS_camera_id << "."
<< std::endl;
return -1;
}
} else {
capture.open(FLAGS_video_path);
if (!capture.isOpened()) {
std::cout << "Can not open the video "
<< FLAGS_video_path << "."
<< std::endl;
return -1;
}
}
// Create a VideoWriter
cv::VideoWriter video_out;
std::string video_out_path;
if (FLAGS_save_result) {
// Get video information: resolution, fps
int video_width = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
int video_height = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));
int video_fps = static_cast<int>(capture.get(CV_CAP_PROP_FPS));
int video_fourcc;
if (FLAGS_use_camera) {
video_fourcc = 828601953;
} else {
video_fourcc = static_cast<int>(capture.get(CV_CAP_PROP_FOURCC));
}
if (FLAGS_use_camera) {
time_t now = time(0);
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir,
std::to_string(now) + ".mp4");
} else {
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_video_path);
}
video_out.open(video_out_path.c_str(),
video_fourcc,
video_fps,
cv::Size(video_width, video_height),
true);
if (!video_out.isOpened()) {
std::cout << "Create video writer failed!" << std::endl;
return -1;
}
}
PaddleX::ClsResult result;
cv::Mat frame;
int key;
while (capture.read(frame)) {
if (FLAGS_show_result || FLAGS_use_camera) {
key = cv::waitKey(1);
// When pressing `ESC`, then exit program and result video is saved
if (key == 27) {
break;
}
} else if (frame.empty()) {
break;
}
// Begin to predict
model.predict(frame, &result);
// Visualize results
cv::Mat vis_img = frame.clone();
auto colormap = PaddleX::GenerateColorMap(model.labels.size());
int c1 = colormap[3 * result.category_id + 0];
int c2 = colormap[3 * result.category_id + 1];
int c3 = colormap[3 * result.category_id + 2];
cv::Scalar text_color = cv::Scalar(c1, c2, c3);
std::string text = result.category;
text += std::to_string(static_cast<int>(result.score * 100)) + "%";
int font_face = cv::FONT_HERSHEY_SIMPLEX;
double font_scale = 0.5f;
float thickness = 0.5;
cv::Size text_size =
cv::getTextSize(text, font_face, font_scale, thickness, nullptr);
cv::Point origin;
origin.x = frame.cols / 2;
origin.y = frame.rows / 2;
cv::Rect text_back = cv::Rect(origin.x,
origin.y - text_size.height,
text_size.width,
text_size.height);
cv::rectangle(vis_img, text_back, text_color, -1);
cv::putText(vis_img,
text,
origin,
font_face,
font_scale,
cv::Scalar(255, 255, 255),
thickness);
if (FLAGS_show_result || FLAGS_use_camera) {
cv::imshow("video_classifier", vis_img);
}
if (FLAGS_save_result) {
video_out.write(vis_img);
}
std::cout << "Predict label: " << result.category
<< ", label_id:" << result.category_id
<< ", score: " << result.score << std::endl;
}
capture.release();
if (FLAGS_save_result) {
video_out.release();
std::cout << "Visualized output saved as " << video_out_path << std::endl;
}
if (FLAGS_show_result || FLAGS_use_camera) {
cv::destroyAllWindows();
}
return 0;
}
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <glog/logging.h>
#include <omp.h>
#include <algorithm>
#include <chrono> // NOLINT
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <utility>
#include "include/paddlex/paddlex.h"
#include "include/paddlex/visualize.h"
#if defined(__arm__) || defined(__aarch64__)
#include <opencv2/videoio/legacy/constants_c.h>
#endif
using namespace std::chrono; // NOLINT
DEFINE_string(model_dir, "", "Path of inference model");
DEFINE_bool(use_gpu, false, "Infering with GPU or CPU");
DEFINE_bool(use_trt, false, "Infering with TensorRT");
DEFINE_int32(gpu_id, 0, "GPU card id");
DEFINE_bool(use_camera, false, "Infering with Camera");
DEFINE_int32(camera_id, 0, "Camera id");
DEFINE_string(video_path, "", "Path of input video");
DEFINE_bool(show_result, false, "show the result of each frame with a window");
DEFINE_bool(save_result, true, "save the result of each frame to a video");
DEFINE_string(key, "", "key of encryption");
DEFINE_string(save_dir, "output", "Path to save visualized image");
DEFINE_double(threshold,
0.5,
"The minimum scores of target boxes which are shown");
int main(int argc, char** argv) {
// Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") {
std::cerr << "--model_dir need to be defined" << std::endl;
return -1;
}
if (FLAGS_video_path == "" & FLAGS_use_camera == false) {
std::cerr << "--video_path or --use_camera need to be defined" << std::endl;
return -1;
}
// Load model
PaddleX::Model model;
model.Init(FLAGS_model_dir,
FLAGS_use_gpu,
FLAGS_use_trt,
FLAGS_gpu_id,
FLAGS_key);
// Open video
cv::VideoCapture capture;
if (FLAGS_use_camera) {
capture.open(FLAGS_camera_id);
if (!capture.isOpened()) {
std::cout << "Can not open the camera "
<< FLAGS_camera_id << "."
<< std::endl;
return -1;
}
} else {
capture.open(FLAGS_video_path);
if (!capture.isOpened()) {
std::cout << "Can not open the video "
<< FLAGS_video_path << "."
<< std::endl;
return -1;
}
}
// Create a VideoWriter
cv::VideoWriter video_out;
std::string video_out_path;
if (FLAGS_save_result) {
// Get video information: resolution, fps
int video_width = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
int video_height = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));
int video_fps = static_cast<int>(capture.get(CV_CAP_PROP_FPS));
int video_fourcc;
if (FLAGS_use_camera) {
video_fourcc = 828601953;
} else {
video_fourcc = static_cast<int>(capture.get(CV_CAP_PROP_FOURCC));
}
if (FLAGS_use_camera) {
time_t now = time(0);
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir,
std::to_string(now) + ".mp4");
} else {
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_video_path);
}
video_out.open(video_out_path.c_str(),
video_fourcc,
video_fps,
cv::Size(video_width, video_height),
true);
if (!video_out.isOpened()) {
std::cout << "Create video writer failed!" << std::endl;
return -1;
}
}
PaddleX::DetResult result;
cv::Mat frame;
int key;
while (capture.read(frame)) {
if (FLAGS_show_result || FLAGS_use_camera) {
key = cv::waitKey(1);
// When pressing `ESC`, then exit program and result video is saved
if (key == 27) {
break;
}
} else if (frame.empty()) {
break;
}
// Begin to predict
model.predict(frame, &result);
// Visualize results
cv::Mat vis_img =
PaddleX::Visualize(frame, result, model.labels, FLAGS_threshold);
if (FLAGS_show_result || FLAGS_use_camera) {
cv::imshow("video_detector", vis_img);
}
if (FLAGS_save_result) {
video_out.write(vis_img);
}
result.clear();
}
capture.release();
if (FLAGS_save_result) {
std::cout << "Visualized output saved as " << video_out_path << std::endl;
video_out.release();
}
if (FLAGS_show_result || FLAGS_use_camera) {
cv::destroyAllWindows();
}
return 0;
}
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <glog/logging.h>
#include <omp.h>
#include <algorithm>
#include <chrono> // NOLINT
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <utility>
#include <ctime>
#include "include/paddlex/paddlex.h"
#include "include/paddlex/visualize.h"
#if defined(__arm__) || defined(__aarch64__)
#include <opencv2/videoio/legacy/constants_c.h>
#endif
using namespace std::chrono; // NOLINT
DEFINE_string(model_dir, "", "Path of inference model");
DEFINE_bool(use_gpu, false, "Infering with GPU or CPU");
DEFINE_bool(use_trt, false, "Infering with TensorRT");
DEFINE_int32(gpu_id, 0, "GPU card id");
DEFINE_string(key, "", "key of encryption");
DEFINE_bool(use_camera, false, "Infering with Camera");
DEFINE_int32(camera_id, 0, "Camera id");
DEFINE_string(video_path, "", "Path of input video");
DEFINE_bool(show_result, false, "show the result of each frame with a window");
DEFINE_bool(save_result, true, "save the result of each frame to a video");
DEFINE_string(save_dir, "output", "Path to save visualized image");
int main(int argc, char** argv) {
// Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") {
std::cerr << "--model_dir need to be defined" << std::endl;
return -1;
}
if (FLAGS_video_path == "" & FLAGS_use_camera == false) {
std::cerr << "--video_path or --use_camera need to be defined" << std::endl;
return -1;
}
// Load model
PaddleX::Model model;
model.Init(FLAGS_model_dir,
FLAGS_use_gpu,
FLAGS_use_trt,
FLAGS_gpu_id,
FLAGS_key);
// Open video
cv::VideoCapture capture;
if (FLAGS_use_camera) {
capture.open(FLAGS_camera_id);
if (!capture.isOpened()) {
std::cout << "Can not open the camera "
<< FLAGS_camera_id << "."
<< std::endl;
return -1;
}
} else {
capture.open(FLAGS_video_path);
if (!capture.isOpened()) {
std::cout << "Can not open the video "
<< FLAGS_video_path << "."
<< std::endl;
return -1;
}
}
// Create a VideoWriter
cv::VideoWriter video_out;
std::string video_out_path;
if (FLAGS_save_result) {
// Get video information: resolution, fps
int video_width = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
int video_height = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));
int video_fps = static_cast<int>(capture.get(CV_CAP_PROP_FPS));
int video_fourcc;
if (FLAGS_use_camera) {
video_fourcc = 828601953;
} else {
video_fourcc = static_cast<int>(capture.get(CV_CAP_PROP_FOURCC));
}
if (FLAGS_use_camera) {
time_t now = time(0);
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir,
std::to_string(now) + ".mp4");
} else {
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_video_path);
}
video_out.open(video_out_path.c_str(),
video_fourcc,
video_fps,
cv::Size(video_width, video_height),
true);
if (!video_out.isOpened()) {
std::cout << "Create video writer failed!" << std::endl;
return -1;
}
}
PaddleX::SegResult result;
cv::Mat frame;
int key;
while (capture.read(frame)) {
if (FLAGS_show_result || FLAGS_use_camera) {
key = cv::waitKey(1);
// When pressing `ESC`, then exit program and result video is saved
if (key == 27) {
break;
}
} else if (frame.empty()) {
break;
}
// Begin to predict
model.predict(frame, &result);
// Visualize results
cv::Mat vis_img = PaddleX::Visualize(frame, result, model.labels);
if (FLAGS_show_result || FLAGS_use_camera) {
cv::imshow("video_segmenter", vis_img);
}
if (FLAGS_save_result) {
video_out.write(vis_img);
}
result.clear();
}
capture.release();
if (FLAGS_save_result) {
video_out.release();
std::cout << "Visualized output saved as " << video_out_path << std::endl;
}
if (FLAGS_show_result || FLAGS_use_camera) {
cv::destroyAllWindows();
}
return 0;
}
...@@ -23,9 +23,9 @@ ...@@ -23,9 +23,9 @@
#else // Linux/Unix #else // Linux/Unix
#include <dirent.h> #include <dirent.h>
// #include <sys/io.h> // #include <sys/io.h>
#ifdef __arm__ // for arm #if defined(__arm__) || defined(__aarch64__) // for arm
#include <aarch64-linux-gpu/sys/stat.h> #include <aarch64-linux-gnu/sys/stat.h>
#include <aarch64-linux-gpu/sys/types.h> #include <aarch64-linux-gnu/sys/types.h>
#else #else
#include <sys/stat.h> #include <sys/stat.h>
#include <sys/types.h> #include <sys/types.h>
......
...@@ -7,12 +7,12 @@ if [ ! -d "./paddlex-encryption" ]; then ...@@ -7,12 +7,12 @@ if [ ! -d "./paddlex-encryption" ]; then
fi fi
# download pre-compiled opencv lib # download pre-compiled opencv lib
OPENCV_URL=https://paddleseg.bj.bcebos.com/deploy/docker/opencv3gcc4.8.tar.bz2 OPENCV_URL=https://bj.bcebos.com/paddleseg/deploy/opencv3.4.6gcc4.8ffmpeg.tar.gz2
if [ ! -d "./deps/opencv3gcc4.8" ]; then if [ ! -d "./deps/opencv3.4.6gcc4.8ffmpeg/" ]; then
mkdir -p deps mkdir -p deps
cd deps cd deps
wget -c ${OPENCV_URL} wget -c ${OPENCV_URL}
tar xvfj opencv3gcc4.8.tar.bz2 tar xvfj opencv3.4.6gcc4.8ffmpeg.tar.gz2
rm -rf opencv3gcc4.8.tar.bz2 rm -rf opencv3.4.6gcc4.8ffmpeg.tar.gz2
cd .. cd ..
fi fi
...@@ -24,7 +24,7 @@ ENCRYPTION_DIR=$(pwd)/paddlex-encryption ...@@ -24,7 +24,7 @@ ENCRYPTION_DIR=$(pwd)/paddlex-encryption
# OPENCV 路径, 如果使用自带预编译版本可不修改 # OPENCV 路径, 如果使用自带预编译版本可不修改
sh $(pwd)/scripts/bootstrap.sh # 下载预编译版本的opencv sh $(pwd)/scripts/bootstrap.sh # 下载预编译版本的opencv
OPENCV_DIR=$(pwd)/deps/opencv3gcc4.8/ OPENCV_DIR=$(pwd)/deps/opencv3.4.6gcc4.8ffmpeg/
# 以下无需改动 # 以下无需改动
rm -rf build rm -rf build
...@@ -42,4 +42,4 @@ cmake .. \ ...@@ -42,4 +42,4 @@ cmake .. \
-DCUDNN_LIB=${CUDNN_LIB} \ -DCUDNN_LIB=${CUDNN_LIB} \
-DENCRYPTION_DIR=${ENCRYPTION_DIR} \ -DENCRYPTION_DIR=${ENCRYPTION_DIR} \
-DOPENCV_DIR=${OPENCV_DIR} -DOPENCV_DIR=${OPENCV_DIR}
make make -j16
# download pre-compiled opencv lib
OPENCV_URL=https://bj.bcebos.com/paddlex/deploy/tools/opencv3_aarch.tgz
if [ ! -d "./deps/opencv3" ]; then
mkdir -p deps
cd deps
wget -c ${OPENCV_URL}
tar xvfz opencv3_aarch.tgz
rm -rf opencv3_aarch.tgz
cd ..
fi
...@@ -14,14 +14,7 @@ WITH_STATIC_LIB=OFF ...@@ -14,14 +14,7 @@ WITH_STATIC_LIB=OFF
# CUDA 的 lib 路径 # CUDA 的 lib 路径
CUDA_LIB=/usr/local/cuda/lib64 CUDA_LIB=/usr/local/cuda/lib64
# CUDNN 的 lib 路径 # CUDNN 的 lib 路径
CUDNN_LIB=/usr/local/cuda/lib64 CUDNN_LIB=/usr/lib/aarch64-linux-gnu
# 是否加载加密后的模型
WITH_ENCRYPTION=OFF
# OPENCV 路径, 如果使用自带预编译版本可不修改
sh $(pwd)/scripts/jetson_bootstrap.sh # 下载预编译版本的opencv
OPENCV_DIR=$(pwd)/deps/opencv3
# 以下无需改动 # 以下无需改动
rm -rf build rm -rf build
...@@ -31,12 +24,9 @@ cmake .. \ ...@@ -31,12 +24,9 @@ cmake .. \
-DWITH_GPU=${WITH_GPU} \ -DWITH_GPU=${WITH_GPU} \
-DWITH_MKL=${WITH_MKL} \ -DWITH_MKL=${WITH_MKL} \
-DWITH_TENSORRT=${WITH_TENSORRT} \ -DWITH_TENSORRT=${WITH_TENSORRT} \
-DWITH_ENCRYPTION=${WITH_ENCRYPTION} \
-DTENSORRT_DIR=${TENSORRT_DIR} \ -DTENSORRT_DIR=${TENSORRT_DIR} \
-DPADDLE_DIR=${PADDLE_DIR} \ -DPADDLE_DIR=${PADDLE_DIR} \
-DWITH_STATIC_LIB=${WITH_STATIC_LIB} \ -DWITH_STATIC_LIB=${WITH_STATIC_LIB} \
-DCUDA_LIB=${CUDA_LIB} \ -DCUDA_LIB=${CUDA_LIB} \
-DCUDNN_LIB=${CUDNN_LIB} \ -DCUDNN_LIB=${CUDNN_LIB}
-DENCRYPTION_DIR=${ENCRYPTION_DIR} \
-DOPENCV_DIR=${OPENCV_DIR}
make make
...@@ -65,7 +65,11 @@ void Model::create_predictor(const std::string& model_dir, ...@@ -65,7 +65,11 @@ void Model::create_predictor(const std::string& model_dir,
config.SwitchUseFeedFetchOps(false); config.SwitchUseFeedFetchOps(false);
config.SwitchSpecifyInputNames(true); config.SwitchSpecifyInputNames(true);
// 开启图优化 // 开启图优化
#if defined(__arm__) || defined(__aarch64__)
config.SwitchIrOptim(false);
#else
config.SwitchIrOptim(use_ir_optim); config.SwitchIrOptim(use_ir_optim);
#endif
// 开启内存优化 // 开启内存优化
config.EnableMemoryOptim(); config.EnableMemoryOptim();
if (use_trt) { if (use_trt) {
......
...@@ -15,6 +15,7 @@ ...@@ -15,6 +15,7 @@
#include <iostream> #include <iostream>
#include <string> #include <string>
#include <vector> #include <vector>
#include <math.h>
#include "include/paddlex/transforms.h" #include "include/paddlex/transforms.h"
...@@ -60,8 +61,8 @@ bool ResizeByShort::Run(cv::Mat* im, ImageBlob* data) { ...@@ -60,8 +61,8 @@ bool ResizeByShort::Run(cv::Mat* im, ImageBlob* data) {
data->reshape_order_.push_back("resize"); data->reshape_order_.push_back("resize");
float scale = GenerateScale(*im); float scale = GenerateScale(*im);
int width = static_cast<int>(scale * im->cols); int width = static_cast<int>(round(scale * im->cols));
int height = static_cast<int>(scale * im->rows); int height = static_cast<int>(round(scale * im->rows));
cv::resize(*im, *im, cv::Size(width, height), 0, 0, cv::INTER_LINEAR); cv::resize(*im, *im, cv::Size(width, height), 0, 0, cv::INTER_LINEAR);
data->new_im_size_[0] = im->rows; data->new_im_size_[0] = im->rows;
......
...@@ -23,6 +23,7 @@ import org.opencv.core.Scalar; ...@@ -23,6 +23,7 @@ import org.opencv.core.Scalar;
import org.opencv.core.Size; import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc; import org.opencv.imgproc.Imgproc;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap; import java.util.HashMap;
import java.util.List; import java.util.List;
...@@ -101,6 +102,15 @@ public class Transforms { ...@@ -101,6 +102,15 @@ public class Transforms {
if (info.containsKey("coarsest_stride")) { if (info.containsKey("coarsest_stride")) {
padding.coarsest_stride = (int) info.get("coarsest_stride"); padding.coarsest_stride = (int) info.get("coarsest_stride");
} }
if (info.containsKey("im_padding_value")) {
List<Double> im_padding_value = (List<Double>) info.get("im_padding_value");
if (im_padding_value.size()!=3){
Log.e(TAG, "len of im_padding_value in padding must == 3.");
}
for (int k =0; i<im_padding_value.size(); i++){
padding.paddding_value[k] = im_padding_value.get(k);
}
}
if (info.containsKey("target_size")) { if (info.containsKey("target_size")) {
if (info.get("target_size") instanceof Integer) { if (info.get("target_size") instanceof Integer) {
padding.width = (int) info.get("target_size"); padding.width = (int) info.get("target_size");
...@@ -124,7 +134,7 @@ public class Transforms { ...@@ -124,7 +134,7 @@ public class Transforms {
if(transformsMode.equalsIgnoreCase("RGB")){ if(transformsMode.equalsIgnoreCase("RGB")){
Imgproc.cvtColor(inputMat, inputMat, Imgproc.COLOR_BGR2RGB); Imgproc.cvtColor(inputMat, inputMat, Imgproc.COLOR_BGR2RGB);
}else if(!transformsMode.equalsIgnoreCase("BGR")){ }else if(!transformsMode.equalsIgnoreCase("BGR")){
Log.e(TAG, "transformsMode only support RGB or BGR"); Log.e(TAG, "transformsMode only support RGB or BGR.");
} }
inputMat.convertTo(inputMat, CvType.CV_32FC(3)); inputMat.convertTo(inputMat, CvType.CV_32FC(3));
...@@ -136,16 +146,15 @@ public class Transforms { ...@@ -136,16 +146,15 @@ public class Transforms {
int h = inputMat.height(); int h = inputMat.height();
int c = inputMat.channels(); int c = inputMat.channels();
imageBlob.setImageData(new float[w * h * c]); imageBlob.setImageData(new float[w * h * c]);
int[] channelStride = new int[]{w * h, w * h * 2};
for (int y = 0; y < h; y++) { Mat singleChannelMat = new Mat(h, w, CvType.CV_32FC(1));
for (int x = 0; float[] singleChannelImageData = new float[w * h];
x < w; x++) { for (int i = 0; i < c; i++) {
double[] color = inputMat.get(y, x); Core.extractChannel(inputMat, singleChannelMat, i);
imageBlob.getImageData()[y * w + x] = (float) (color[0]); singleChannelMat.get(0, 0, singleChannelImageData);
imageBlob.getImageData()[y * w + x + channelStride[0]] = (float) (color[1]); System.arraycopy(singleChannelImageData ,0, imageBlob.getImageData(),i*w*h, w*h);
imageBlob.getImageData()[y * w + x + channelStride[1]] = (float) (color[2]);
}
} }
return imageBlob; return imageBlob;
} }
...@@ -248,6 +257,7 @@ public class Transforms { ...@@ -248,6 +257,7 @@ public class Transforms {
private double width; private double width;
private double height; private double height;
private double coarsest_stride; private double coarsest_stride;
private double[] paddding_value = {0.0, 0.0, 0.0};
public Mat run(Mat inputMat, ImageBlob imageBlob) { public Mat run(Mat inputMat, ImageBlob imageBlob) {
int origin_w = inputMat.width(); int origin_w = inputMat.width();
...@@ -264,7 +274,7 @@ public class Transforms { ...@@ -264,7 +274,7 @@ public class Transforms {
} }
imageBlob.setNewImageSize(inputMat.height(),2); imageBlob.setNewImageSize(inputMat.height(),2);
imageBlob.setNewImageSize(inputMat.width(),3); imageBlob.setNewImageSize(inputMat.width(),3);
Core.copyMakeBorder(inputMat, inputMat, 0, (int)padding_h, 0, (int)padding_w, Core.BORDER_CONSTANT, new Scalar(0)); Core.copyMakeBorder(inputMat, inputMat, 0, (int)padding_h, 0, (int)padding_w, Core.BORDER_CONSTANT, new Scalar(paddding_value));
return inputMat; return inputMat;
} }
} }
......
...@@ -31,8 +31,11 @@ import org.opencv.core.Scalar; ...@@ -31,8 +31,11 @@ import org.opencv.core.Scalar;
import org.opencv.core.Size; import org.opencv.core.Size;
import org.opencv.imgproc.Imgproc; import org.opencv.imgproc.Imgproc;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Arrays; import java.util.Arrays;
import java.util.Date;
import java.util.List; import java.util.List;
import java.util.ListIterator; import java.util.ListIterator;
import java.util.Map; import java.util.Map;
...@@ -118,13 +121,13 @@ public class Visualize { ...@@ -118,13 +121,13 @@ public class Visualize {
public Mat draw(SegResult result, Mat visualizeMat, ImageBlob imageBlob, int cutoutClass) { public Mat draw(SegResult result, Mat visualizeMat, ImageBlob imageBlob, int cutoutClass) {
int new_h = (int)imageBlob.getNewImageSize()[2]; int new_h = (int)imageBlob.getNewImageSize()[2];
int new_w = (int)imageBlob.getNewImageSize()[3]; int new_w = (int)imageBlob.getNewImageSize()[3];
Mat mask = new Mat(new_h, new_w, CvType.CV_8UC(1)); Mat mask = new Mat(new_h, new_w, CvType.CV_32FC(1));
float[] scoreData = new float[new_h*new_w];
System.arraycopy(result.getMask().getScoreData() ,cutoutClass*new_h*new_w, scoreData ,0, new_h*new_w);
mask.put(0,0, scoreData);
Core.multiply(mask, new Scalar(255), mask);
mask.convertTo(mask,CvType.CV_8UC(1));
for (int h = 0; h < new_h; h++) {
for (int w = 0; w < new_w; w++){
mask.put(h , w, (1-result.getMask().getScoreData()[cutoutClass + h * new_h + w]) * 255);
}
}
ListIterator<Map.Entry<String, int[]>> reverseReshapeInfo = new ArrayList<Map.Entry<String, int[]>>(imageBlob.getReshapeInfo().entrySet()).listIterator(imageBlob.getReshapeInfo().size()); ListIterator<Map.Entry<String, int[]>> reverseReshapeInfo = new ArrayList<Map.Entry<String, int[]>>(imageBlob.getReshapeInfo().entrySet()).listIterator(imageBlob.getReshapeInfo().size());
while (reverseReshapeInfo.hasPrevious()) { while (reverseReshapeInfo.hasPrevious()) {
Map.Entry<String, int[]> entry = reverseReshapeInfo.previous(); Map.Entry<String, int[]> entry = reverseReshapeInfo.previous();
...@@ -135,10 +138,7 @@ public class Visualize { ...@@ -135,10 +138,7 @@ public class Visualize {
Size sz = new Size(entry.getValue()[0], entry.getValue()[1]); Size sz = new Size(entry.getValue()[0], entry.getValue()[1]);
Imgproc.resize(mask, mask, sz,0,0,Imgproc.INTER_LINEAR); Imgproc.resize(mask, mask, sz,0,0,Imgproc.INTER_LINEAR);
} }
Log.i(TAG, "postprocess operator: " + entry.getKey());
Log.i(TAG, "shape:: " + String.valueOf(mask.width()) + ","+ String.valueOf(mask.height()));
} }
Mat dst = new Mat(); Mat dst = new Mat();
List<Mat> listMat = Arrays.asList(visualizeMat, mask); List<Mat> listMat = Arrays.asList(visualizeMat, mask);
Core.merge(listMat, dst); Core.merge(listMat, dst);
......
#copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. #copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
#Licensed under the Apache License, Version 2.0 (the "License"); #Licensed under the Apache License, Version 2.0 (the "License");
#you may not use this file except in compliance with the License. #you may not use this file except in compliance with the License.
......
# PaddleX文档 # PaddleX文档
PaddleX的使用文档均在本目录结构下。文档采用Read the Docs方式组织,您可以直接访问[在线文档](https://paddlex.readthedocs.io/zh_CN/latest/index.html)进行查阅。 PaddleX的使用文档均在本目录结构下。文档采用Read the Docs方式组织,您可以直接访问[在线文档](https://paddlex.readthedocs.io/zh_CN/develop/index.html)进行查阅。
## 编译文档 ## 编译文档
在本目录下按如下步骤进行文档编译 在本目录下按如下步骤进行文档编译
......
...@@ -5,7 +5,7 @@ ...@@ -5,7 +5,7 @@
``` ```
paddlex.datasets.ImageNet(data_dir, file_list, label_list, transforms=None, num_workers=‘auto’, buffer_size=8, parallel_method='process', shuffle=False) paddlex.datasets.ImageNet(data_dir, file_list, label_list, transforms=None, num_workers=‘auto’, buffer_size=8, parallel_method='process', shuffle=False)
``` ```
读取ImageNet格式的分类数据集,并对样本进行相应的处理。ImageNet数据集格式的介绍可查看文档:[数据集格式说明](../data/format/index.html) 读取ImageNet格式的分类数据集,并对样本进行相应的处理。ImageNet数据集格式的介绍可查看文档:[数据集格式说明](../data/format/classification.md)
示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/image_classification/mobilenetv2.py) 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/image_classification/mobilenetv2.py)
...@@ -26,7 +26,7 @@ paddlex.datasets.ImageNet(data_dir, file_list, label_list, transforms=None, num_ ...@@ -26,7 +26,7 @@ paddlex.datasets.ImageNet(data_dir, file_list, label_list, transforms=None, num_
paddlex.datasets.VOCDetection(data_dir, file_list, label_list, transforms=None, num_workers=‘auto’, buffer_size=100, parallel_method='process', shuffle=False) paddlex.datasets.VOCDetection(data_dir, file_list, label_list, transforms=None, num_workers=‘auto’, buffer_size=100, parallel_method='process', shuffle=False)
``` ```
> 读取PascalVOC格式的检测数据集,并对样本进行相应的处理。PascalVOC数据集格式的介绍可查看文档:[数据集格式说明](../data/format/index.html) > 读取PascalVOC格式的检测数据集,并对样本进行相应的处理。PascalVOC数据集格式的介绍可查看文档:[数据集格式说明](../data/format/detection.md)
> 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_darknet53.py) > 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_darknet53.py)
...@@ -47,7 +47,7 @@ paddlex.datasets.VOCDetection(data_dir, file_list, label_list, transforms=None, ...@@ -47,7 +47,7 @@ paddlex.datasets.VOCDetection(data_dir, file_list, label_list, transforms=None,
paddlex.datasets.CocoDetection(data_dir, ann_file, transforms=None, num_workers='auto', buffer_size=100, parallel_method='process', shuffle=False) paddlex.datasets.CocoDetection(data_dir, ann_file, transforms=None, num_workers='auto', buffer_size=100, parallel_method='process', shuffle=False)
``` ```
> 读取MSCOCO格式的检测数据集,并对样本进行相应的处理,该格式的数据集同样可以应用到实例分割模型的训练中。MSCOCO数据集格式的介绍可查看文档:[数据集格式说明](../data/format/index.html) > 读取MSCOCO格式的检测数据集,并对样本进行相应的处理,该格式的数据集同样可以应用到实例分割模型的训练中。MSCOCO数据集格式的介绍可查看文档:[数据集格式说明](../data/format/instance_segmentation.md)
> 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/instance_segmentation/mask_rcnn_r50_fpn.py) > 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/instance_segmentation/mask_rcnn_r50_fpn.py)
...@@ -67,7 +67,7 @@ paddlex.datasets.CocoDetection(data_dir, ann_file, transforms=None, num_workers= ...@@ -67,7 +67,7 @@ paddlex.datasets.CocoDetection(data_dir, ann_file, transforms=None, num_workers=
paddlex.datasets.SegDataset(data_dir, file_list, label_list, transforms=None, num_workers='auto', buffer_size=100, parallel_method='process', shuffle=False) paddlex.datasets.SegDataset(data_dir, file_list, label_list, transforms=None, num_workers='auto', buffer_size=100, parallel_method='process', shuffle=False)
``` ```
> 读取语义分割任务数据集,并对样本进行相应的处理。语义分割任务数据集格式的介绍可查看文档:[数据集格式说明](../data/format/index.html) > 读取语义分割任务数据集,并对样本进行相应的处理。语义分割任务数据集格式的介绍可查看文档:[数据集格式说明](../data/format/segmentation.md)
> 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/unet.py) > 示例:[代码文件](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/unet.py)
......
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
图像分类、目标检测、实例分割、语义分割统一的预测器,实现高性能预测。 图像分类、目标检测、实例分割、语义分割统一的预测器,实现高性能预测。
``` ```
paddlex.deploy.Predictor(model_dir, use_gpu=False, gpu_id=0, use_mkl=False, use_trt=False, use_glog=False, memory_optimize=True) paddlex.deploy.Predictor(model_dir, use_gpu=False, gpu_id=0, use_mkl=False, mkl_thread_num=4, use_trt=False, use_glog=False, memory_optimize=True)
``` ```
**参数** **参数**
...@@ -16,6 +16,7 @@ paddlex.deploy.Predictor(model_dir, use_gpu=False, gpu_id=0, use_mkl=False, use_ ...@@ -16,6 +16,7 @@ paddlex.deploy.Predictor(model_dir, use_gpu=False, gpu_id=0, use_mkl=False, use_
> * **use_gpu** (bool): 是否使用GPU进行预测。 > * **use_gpu** (bool): 是否使用GPU进行预测。
> * **gpu_id** (int): 使用的GPU序列号。 > * **gpu_id** (int): 使用的GPU序列号。
> * **use_mkl** (bool): 是否使用mkldnn加速库。 > * **use_mkl** (bool): 是否使用mkldnn加速库。
> * **mkl_thread_num** (int): 使用mkldnn加速库时的线程数,默认为4
> * **use_trt** (boll): 是否使用TensorRT预测引擎。 > * **use_trt** (boll): 是否使用TensorRT预测引擎。
> * **use_glog** (bool): 是否打印中间日志。 > * **use_glog** (bool): 是否打印中间日志。
> * **memory_optimize** (bool): 是否优化内存使用。 > * **memory_optimize** (bool): 是否优化内存使用。
...@@ -40,7 +41,7 @@ predict(image, topk=1) ...@@ -40,7 +41,7 @@ predict(image, topk=1)
> **参数** > **参数**
> >
> > * **image** (str|np.ndarray): 待预测的图片路径或numpy数组(HWC排列,BGR格式)。 > > * **image** (str|np.ndarray): 待预测的图片路径或numpy数组(HWC排列,BGR格式)。
> > * **topk** (int): 图像分类时使用的参数,表示预测前topk个可能的分类 > > * **topk** (int): 图像分类时使用的参数,表示预测前topk个可能的分类
### batch_predict 接口 ### batch_predict 接口
``` ```
......
# Object Detection # Object Detection
## paddlex.det.PPYOLO
```python
paddlex.det.PPYOLO(num_classes=80, backbone='ResNet50_vd_ssld', with_dcn_v2=True, anchors=None, anchor_masks=None, use_coord_conv=True, use_iou_aware=True, use_spp=True, use_drop_block=True, scale_x_y=1.05, ignore_threshold=0.7, label_smooth=False, use_iou_loss=True, use_matrix_nms=True, nms_score_threshold=0.01, nms_topk=1000, nms_keep_topk=100, nms_iou_threshold=0.45, train_random_shapes=[320, 352, 384, 416, 448, 480, 512, 544, 576, 608])
```
> 构建PPYOLO检测器。**注意在PPYOLO,num_classes不需要包含背景类,如目标包括human、dog两种,则num_classes设为2即可,这里与FasterRCNN/MaskRCNN有差别**
> **参数**
>
> > - **num_classes** (int): 类别数。默认为80。
> > - **backbone** (str): PPYOLO的backbone网络,取值范围为['ResNet50_vd_ssld']。默认为'ResNet50_vd_ssld'。
> > - **with_dcn_v2** (bool): Backbone是否使用DCNv2结构。默认为True。
> > - **anchors** (list|tuple): anchor框的宽度和高度,为None时表示使用默认值
> > [[10, 13], [16, 30], [33, 23], [30, 61], [62, 45],
> [59, 119], [116, 90], [156, 198], [373, 326]]。
> > - **anchor_masks** (list|tuple): 在计算PPYOLO损失时,使用anchor的mask索引,为None时表示使用默认值
> > [[6, 7, 8], [3, 4, 5], [0, 1, 2]]。
> > - **use_coord_conv** (bool): 是否使用CoordConv。默认值为True。
> > - **use_iou_aware** (bool): 是否使用IoU Aware分支。默认值为True。
> > - **use_spp** (bool): 是否使用Spatial Pyramid Pooling结构。默认值为True。
> > - **use_drop_block** (bool): 是否使用Drop Block。默认值为True。
> > - **scale_x_y** (float): 调整中心点位置时的系数因子。默认值为1.05。
> > - **use_iou_loss** (bool): 是否使用IoU loss。默认值为True。
> > - **use_matrix_nms** (bool): 是否使用Matrix NMS。默认值为True。
> > - **ignore_threshold** (float): 在计算PPYOLO损失时,IoU大于`ignore_threshold`的预测框的置信度被忽略。默认为0.7。
> > - **nms_score_threshold** (float): 检测框的置信度得分阈值,置信度得分低于阈值的框应该被忽略。默认为0.01。
> > - **nms_topk** (int): 进行NMS时,根据置信度保留的最大检测框数。默认为1000。
> > - **nms_keep_topk** (int): 进行NMS后,每个图像要保留的总检测框数。默认为100。
> > - **nms_iou_threshold** (float): 进行NMS时,用于剔除检测框IOU的阈值。默认为0.45。
> > - **label_smooth** (bool): 是否使用label smooth。默认值为False。
> > - **train_random_shapes** (list|tuple): 训练时从列表中随机选择图像大小。默认值为[320, 352, 384, 416, 448, 480, 512, 544, 576, 608]。
### train
```python
train(self, num_epochs, train_dataset, train_batch_size=8, eval_dataset=None, save_interval_epochs=20, log_interval_steps=2, save_dir='output', pretrain_weights='IMAGENET', optimizer=None, learning_rate=1.0/8000, warmup_steps=1000, warmup_start_lr=0.0, lr_decay_epochs=[213, 240], lr_decay_gamma=0.1, metric=None, use_vdl=False, sensitivities_file=None, eval_metric_loss=0.05, early_stop=False, early_stop_patience=5, resume_checkpoint=None, use_ema=True, ema_decay=0.9998)
```
> PPYOLO模型的训练接口,函数内置了`piecewise`学习率衰减策略和`momentum`优化器。
> **参数**
>
> > - **num_epochs** (int): 训练迭代轮数。
> > - **train_dataset** (paddlex.datasets): 训练数据读取器。
> > - **train_batch_size** (int): 训练数据batch大小。目前检测仅支持单卡评估,训练数据batch大小与显卡数量之商为验证数据batch大小。默认值为8。
> > - **eval_dataset** (paddlex.datasets): 验证数据读取器。
> > - **save_interval_epochs** (int): 模型保存间隔(单位:迭代轮数)。默认为20。
> > - **log_interval_steps** (int): 训练日志输出间隔(单位:迭代次数)。默认为2。
> > - **save_dir** (str): 模型保存路径。默认值为'output'。
> > - **pretrain_weights** (str): 若指定为路径时,则加载路径下预训练模型;若为字符串'IMAGENET',则自动下载在ImageNet图片数据上预训练的模型权重;若为字符串'COCO',则自动下载在COCO数据集上预训练的模型权重;若为None,则不使用预训练模型。默认为None。
> > - **optimizer** (paddle.fluid.optimizer): 优化器。当该参数为None时,使用默认优化器:fluid.layers.piecewise_decay衰减策略,fluid.optimizer.Momentum优化方法。
> > - **learning_rate** (float): 默认优化器的学习率。默认为1.0/8000。
> > - **warmup_steps** (int): 默认优化器进行warmup过程的步数。默认为1000。
> > - **warmup_start_lr** (int): 默认优化器warmup的起始学习率。默认为0.0。
> > - **lr_decay_epochs** (list): 默认优化器的学习率衰减轮数。默认为[213, 240]。
> > - **lr_decay_gamma** (float): 默认优化器的学习率衰减率。默认为0.1。
> > - **metric** (bool): 训练过程中评估的方式,取值范围为['COCO', 'VOC']。默认值为None。
> > - **use_vdl** (bool): 是否使用VisualDL进行可视化。默认值为False。
> > - **sensitivities_file** (str): 若指定为路径时,则加载路径下敏感度信息进行裁剪;若为字符串'DEFAULT',则自动下载在PascalVOC数据上获得的敏感度信息进行裁剪;若为None,则不进行裁剪。默认为None。
> > - **eval_metric_loss** (float): 可容忍的精度损失。默认为0.05。
> > - **early_stop** (bool): 是否使用提前终止训练策略。默认值为False。
> > - **early_stop_patience** (int): 当使用提前终止训练策略时,如果验证集精度在`early_stop_patience`个epoch内连续下降或持平,则终止训练。默认值为5。
> > - **resume_checkpoint** (str): 恢复训练时指定上次训练保存的模型路径。若为None,则不会恢复训练。默认值为None。
> > - **use_ema** (bool): 是否使用指数衰减计算参数的滑动平均值。默认值为True。
> > - **ema_decay** (float): 指数衰减率。默认值为0.9998。
### evaluate
```python
evaluate(self, eval_dataset, batch_size=1, epoch_id=None, metric=None, return_details=False)
```
> PPYOLO模型的评估接口,模型评估后会返回在验证集上的指标`box_map`(metric指定为'VOC'时)或`box_mmap`(metric指定为`COCO`时)。
> **参数**
>
> > - **eval_dataset** (paddlex.datasets): 验证数据读取器。
> > - **batch_size** (int): 验证数据批大小。默认为1。
> > - **epoch_id** (int): 当前评估模型所在的训练轮数。
> > - **metric** (bool): 训练过程中评估的方式,取值范围为['COCO', 'VOC']。默认为None,根据用户传入的Dataset自动选择,如为VOCDetection,则`metric`为'VOC';如为COCODetection,则`metric`为'COCO'默认为None, 如为EasyData类型数据集,同时也会使用'VOC'。
> > - **return_details** (bool): 是否返回详细信息。默认值为False。
> >
> **返回值**
>
> > - **tuple** (metrics, eval_details) | **dict** (metrics): 当`return_details`为True时,返回(metrics, eval_details),当`return_details`为False时,返回metrics。metrics为dict,包含关键字:'bbox_mmap'或者’bbox_map‘,分别表示平均准确率平均值在各个阈值下的结果取平均值的结果(mmAP)、平均准确率平均值(mAP)。eval_details为dict,包含关键字:'bbox',对应元素预测结果列表,每个预测结果由图像id、预测框类别id、预测框坐标、预测框得分;’gt‘:真实标注框相关信息。
### predict
```python
predict(self, img_file, transforms=None)
```
> PPYOLO模型预测接口。需要注意的是,只有在训练过程中定义了eval_dataset,模型在保存时才会将预测时的图像处理流程保存在`YOLOv3.test_transforms`和`YOLOv3.eval_transforms`中。如未在训练时定义eval_dataset,那在调用预测`predict`接口时,用户需要再重新定义`test_transforms`传入给`predict`接口
> **参数**
>
> > - **img_file** (str|np.ndarray): 预测图像路径或numpy数组(HWC排列,BGR格式)。
> > - **transforms** (paddlex.det.transforms): 数据预处理操作。
>
> **返回值**
>
> > - **list**: 预测结果列表,列表中每个元素均为一个dict,key包括'bbox', 'category', 'category_id', 'score',分别表示每个预测目标的框坐标信息、类别、类别id、置信度,其中框坐标信息为[xmin, ymin, w, h],即左上角x, y坐标和框的宽和高。
### batch_predict
```python
batch_predict(self, img_file_list, transforms=None, thread_num=2)
```
> PPYOLO模型批量预测接口。需要注意的是,只有在训练过程中定义了eval_dataset,模型在保存时才会将预测时的图像处理流程保存在`YOLOv3.test_transforms`和`YOLOv3.eval_transforms`中。如未在训练时定义eval_dataset,那在调用预测`batch_predict`接口时,用户需要再重新定义`test_transforms`传入给`batch_predict`接口
> **参数**
>
> > - **img_file_list** (str|np.ndarray): 对列表(或元组)中的图像同时进行预测,列表中的元素是预测图像路径或numpy数组(HWC排列,BGR格式)。
> > - **transforms** (paddlex.det.transforms): 数据预处理操作。
> > - **thread_num** (int): 并发执行各图像预处理时的线程数。
>
> **返回值**
>
> > - **list**: 每个元素都为列表,表示各图像的预测结果。在各图像的预测结果列表中,每个元素均为一个dict,key包括'bbox', 'category', 'category_id', 'score',分别表示每个预测目标的框坐标信息、类别、类别id、置信度,其中框坐标信息为[xmin, ymin, w, h],即左上角x, y坐标和框的宽和高。
## paddlex.det.YOLOv3 ## paddlex.det.YOLOv3
```python ```python
...@@ -21,7 +145,7 @@ paddlex.det.YOLOv3(num_classes=80, backbone='MobileNetV1', anchors=None, anchor_ ...@@ -21,7 +145,7 @@ paddlex.det.YOLOv3(num_classes=80, backbone='MobileNetV1', anchors=None, anchor_
> > - **nms_score_threshold** (float): 检测框的置信度得分阈值,置信度得分低于阈值的框应该被忽略。默认为0.01。 > > - **nms_score_threshold** (float): 检测框的置信度得分阈值,置信度得分低于阈值的框应该被忽略。默认为0.01。
> > - **nms_topk** (int): 进行NMS时,根据置信度保留的最大检测框数。默认为1000。 > > - **nms_topk** (int): 进行NMS时,根据置信度保留的最大检测框数。默认为1000。
> > - **nms_keep_topk** (int): 进行NMS后,每个图像要保留的总检测框数。默认为100。 > > - **nms_keep_topk** (int): 进行NMS后,每个图像要保留的总检测框数。默认为100。
> > - **nms_iou_threshold** (float): 进行NMS时,用于剔除检测框IOU的阈值。默认为0.45。 > > - **nms_iou_threshold** (float): 进行NMS时,用于剔除检测框IoU的阈值。默认为0.45。
> > - **label_smooth** (bool): 是否使用label smooth。默认值为False。 > > - **label_smooth** (bool): 是否使用label smooth。默认值为False。
> > - **train_random_shapes** (list|tuple): 训练时从列表中随机选择图像大小。默认值为[320, 352, 384, 416, 448, 480, 512, 544, 576, 608]。 > > - **train_random_shapes** (list|tuple): 训练时从列表中随机选择图像大小。默认值为[320, 352, 384, 416, 448, 480, 512, 544, 576, 608]。
......
...@@ -101,4 +101,4 @@ batch_predict(self, img_file_list, transforms=None, thread_num=2) ...@@ -101,4 +101,4 @@ batch_predict(self, img_file_list, transforms=None, thread_num=2)
> >
> **返回值** > **返回值**
> >
> > - **list**: 每个元素都为列表,表示各图像的预测结果。在各图像的预测结果列表中,每个元素均为一个dict,key'bbox', 'mask', 'category', 'category_id', 'score',分别表示每个预测目标的框坐标信息、Mask信息,类别、类别id、置信度。其中框坐标信息为[xmin, ymin, w, h],即左上角x, y坐标和框的宽和高。Mask信息为原图大小的二值图,1表示像素点属于预测类别,0表示像素点是背景。 > > - **list**: 每个元素都为列表,表示各图像的预测结果。在各图像的预测结果列表中,每个元素均为一个dict,包含关键字:'bbox', 'mask', 'category', 'category_id', 'score',分别表示每个预测目标的框坐标信息、Mask信息,类别、类别id、置信度。其中框坐标信息为[xmin, ymin, w, h],即左上角x, y坐标和框的宽和高。Mask信息为原图大小的二值图,1表示像素点属于预测类别,0表示像素点是背景。
...@@ -3,7 +3,7 @@ ...@@ -3,7 +3,7 @@
## paddlex.seg.DeepLabv3p ## paddlex.seg.DeepLabv3p
```python ```python
paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride=16, aspp_with_sep_conv=True, decoder_use_sep_conv=True, encoder_with_aspp=True, enable_decoder=True, use_bce_loss=False, use_dice_loss=False, class_weight=None, ignore_index=255) paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride=16, aspp_with_sep_conv=True, decoder_use_sep_conv=True, encoder_with_aspp=True, enable_decoder=True, use_bce_loss=False, use_dice_loss=False, class_weight=None, ignore_index=255, pooling_crop_size=None)
``` ```
...@@ -12,7 +12,7 @@ paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride ...@@ -12,7 +12,7 @@ paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride
> **参数** > **参数**
> > - **num_classes** (int): 类别数。 > > - **num_classes** (int): 类别数。
> > - **backbone** (str): DeepLabv3+的backbone网络,实现特征图的计算,取值范围为['Xception65', 'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0'],默认值为'MobileNetV2_x1.0'。 > > - **backbone** (str): DeepLabv3+的backbone网络,实现特征图的计算,取值范围为['Xception65', 'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0', 'MobileNetV3_large_x1_0_ssld'],默认值为'MobileNetV2_x1.0'。
> > - **output_stride** (int): backbone 输出特征图相对于输入的下采样倍数,一般取值为8或16。默认16。 > > - **output_stride** (int): backbone 输出特征图相对于输入的下采样倍数,一般取值为8或16。默认16。
> > - **aspp_with_sep_conv** (bool): decoder模块是否采用separable convolutions。默认True。 > > - **aspp_with_sep_conv** (bool): decoder模块是否采用separable convolutions。默认True。
> > - **decoder_use_sep_conv** (bool): decoder模块是否采用separable convolutions。默认True。 > > - **decoder_use_sep_conv** (bool): decoder模块是否采用separable convolutions。默认True。
...@@ -22,6 +22,7 @@ paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride ...@@ -22,6 +22,7 @@ paddlex.seg.DeepLabv3p(num_classes=2, backbone='MobileNetV2_x1.0', output_stride
> > - **use_dice_loss** (bool): 是否使用dice loss作为网络的损失函数,只能用于两类分割,可与bce loss同时使用,当`use_bce_loss`和`use_dice_loss`都为False时,使用交叉熵损失函数。默认False。 > > - **use_dice_loss** (bool): 是否使用dice loss作为网络的损失函数,只能用于两类分割,可与bce loss同时使用,当`use_bce_loss`和`use_dice_loss`都为False时,使用交叉熵损失函数。默认False。
> > - **class_weight** (list/str): 交叉熵损失函数各类损失的权重。当`class_weight`为list的时候,长度应为`num_classes`。当`class_weight`为str时, weight.lower()应为'dynamic',这时会根据每一轮各类像素的比重自行计算相应的权重,每一类的权重为:每类的比例 * num_classes。class_weight取默认值None是,各类的权重1,即平时使用的交叉熵损失函数。 > > - **class_weight** (list/str): 交叉熵损失函数各类损失的权重。当`class_weight`为list的时候,长度应为`num_classes`。当`class_weight`为str时, weight.lower()应为'dynamic',这时会根据每一轮各类像素的比重自行计算相应的权重,每一类的权重为:每类的比例 * num_classes。class_weight取默认值None是,各类的权重1,即平时使用的交叉熵损失函数。
> > - **ignore_index** (int): label上忽略的值,label为`ignore_index`的像素不参与损失函数的计算。默认255。 > > - **ignore_index** (int): label上忽略的值,label为`ignore_index`的像素不参与损失函数的计算。默认255。
> > - **pooling_crop_size** (int):当backbone为`MobileNetV3_large_x1_0_ssld`时,需设置为训练过程中模型输入大小,格式为[W, H]。例如模型输入大小为[512, 512], 则`pooling_crop_size`应该设置为[512, 512]。在encoder模块中获取图像平均值时被用到,若为None,则直接求平均值;若为模型输入大小,则使用`avg_pool`算子得到平均值。默认值None。
### train ### train
...@@ -69,7 +70,7 @@ evaluate(self, eval_dataset, batch_size=1, epoch_id=None, return_details=False): ...@@ -69,7 +70,7 @@ evaluate(self, eval_dataset, batch_size=1, epoch_id=None, return_details=False):
> **返回值** > **返回值**
> > > >
> > - **dict**: 当`return_details`为False时,返回dict。包含关键字:'miou'、'category_iou'、'macc'、 > > - **dict**: 当`return_details`为False时,返回dict。包含关键字:'miou'、'category_iou'、'macc'、
> > 'category_acc'和'kappa',分别表示平均iou、各类别iou、平均准确率、各类别准确率和kappa系数。 > > 'category_acc'和'kappa',分别表示平均IoU、各类别IoU、平均准确率、各类别准确率和kappa系数。
> > - **tuple** (metrics, eval_details):当`return_details`为True时,增加返回dict (eval_details), > > - **tuple** (metrics, eval_details):当`return_details`为True时,增加返回dict (eval_details),
> > 包含关键字:'confusion_matrix',表示评估的混淆矩阵。 > > 包含关键字:'confusion_matrix',表示评估的混淆矩阵。
......
...@@ -26,16 +26,16 @@ paddlex.slim.cal_params_sensitivities(model, save_file, eval_dataset, batch_size ...@@ -26,16 +26,16 @@ paddlex.slim.cal_params_sensitivities(model, save_file, eval_dataset, batch_size
``` ```
paddlex.slim.export_quant_model(model, test_dataset, batch_size=2, batch_num=10, save_dir='./quant_model', cache_dir='./temp') paddlex.slim.export_quant_model(model, test_dataset, batch_size=2, batch_num=10, save_dir='./quant_model', cache_dir='./temp')
``` ```
导出量化模型,该接口实现了Post Quantization量化方式,需要传入测试数据集,并设定`batch_size``batch_num`。量化过程中会以数量为`batch_size` X `batch_num`的样本数据的计算结果为统计信息完成模型的量化。 导出量化模型,该接口实现了Post Quantization量化方式,需要传入测试数据集,并设定`batch_size``batch_num`。量化过程中会以数量为`batch_size` * `batch_num`的样本数据的计算结果为统计信息完成模型的量化。
**参数** **参数**
* **model**(paddlex.cls.models/paddlex.det.models/paddlex.seg.models): paddlex加载的模型。 * **model**(paddlex.cls.models/paddlex.det.models/paddlex.seg.models): paddlex加载的模型。
* **test_dataset**(paddlex.dataset): 测试数据集 * **test_dataset**(paddlex.dataset): 测试数据集
* **batch_size**(int): 进行前向计算时的批数据大小 * **batch_size**(int): 进行前向计算时的批数据大小
* **batch_num**(int): 进行向前计算时批数据数量 * **batch_num**(int): 进行向前计算时批数据数量
* **save_dir**(str): 量化后模型的保存目录 * **save_dir**(str): 量化后模型的保存目录
* **cache_dir**(str): 量化过程中的统计数据临时存储目录 * **cache_dir**(str): 量化过程中的统计数据临时存储目录
**使用示例** **使用示例**
......
...@@ -10,11 +10,11 @@ PaddleX对于图像分类、目标检测、实例分割和语义分割内置了 ...@@ -10,11 +10,11 @@ PaddleX对于图像分类、目标检测、实例分割和语义分割内置了
| :------- | :------------| | :------- | :------------|
| 图像分类 | [RandomCrop](cls_transforms.html#randomcrop)[RandomHorizontalFlip](cls_transforms.html#randomhorizontalflip)[RandomVerticalFlip](cls_transforms.html#randomverticalflip)<br> [RandomRotate](cls_transforms.html#randomratate)[RandomDistort](cls_transforms.html#randomdistort) | | 图像分类 | [RandomCrop](cls_transforms.html#randomcrop)[RandomHorizontalFlip](cls_transforms.html#randomhorizontalflip)[RandomVerticalFlip](cls_transforms.html#randomverticalflip)<br> [RandomRotate](cls_transforms.html#randomratate)[RandomDistort](cls_transforms.html#randomdistort) |
|目标检测<br>实例分割| [RandomHorizontalFlip](det_transforms.html#randomhorizontalflip)[RandomDistort](det_transforms.html#randomdistort)[RandomCrop](det_transforms.html#randomcrop)<br> [MixupImage](det_transforms.html#mixupimage)(仅支持YOLOv3模型)[RandomExpand](det_transforms.html#randomexpand) | |目标检测<br>实例分割| [RandomHorizontalFlip](det_transforms.html#randomhorizontalflip)[RandomDistort](det_transforms.html#randomdistort)[RandomCrop](det_transforms.html#randomcrop)<br> [MixupImage](det_transforms.html#mixupimage)(仅支持YOLOv3模型)[RandomExpand](det_transforms.html#randomexpand) |
|语义分割 | [RandomHorizontalFlip](seg_transforms.html#randomhorizontalflip)[RandomVerticalFlip](seg_transforms.html#randomverticalflip)[RandomRangeScaling](seg_transforms.html#randomrangescaling)<br> [RandomStepScaling](seg_transforms.html#randomstepscaling)[RandomPaddingCrop](seg_transforms.html#randompaddingcrop)[RandomBlur](seg_transforms.html#randomblur)<br> [RandomRotate](seg_transforms.html#randomrotate)[RandomScaleAspect](seg_transforms.html#randomscaleaspect)[RandomDistort](seg_transforms.html#randomdistort) | |语义分割 | [RandomHorizontalFlip](seg_transforms.html#randomhorizontalflip)[RandomVerticalFlip](seg_transforms.html#randomverticalflip)[ResizeRangeScaling](seg_transforms.html#resizerangescaling)<br> [ResizeStepScaling](seg_transforms.html#resizestepscaling)[RandomPaddingCrop](seg_transforms.html#randompaddingcrop)[RandomBlur](seg_transforms.html#randomblur)<br> [RandomRotate](seg_transforms.html#randomrotate)[RandomScaleAspect](seg_transforms.html#randomscaleaspect)[RandomDistort](seg_transforms.html#randomdistort) |
## imgaug增强库的支持 ## imgaug增强库的支持
PaddleX目前已适配imgaug图像增强库,用户可以直接在PaddleX构造`transforms`时,调用imgaug的方法, 如下示例 PaddleX目前已适配imgaug图像增强库,用户可以直接在PaddleX构造`transforms`时,调用imgaug的方法,如下示例,
``` ```
import paddlex as pdx import paddlex as pdx
from paddlex.cls import transforms from paddlex.cls import transforms
......
...@@ -16,7 +16,7 @@ paddlex.seg.transforms.Compose(transforms) ...@@ -16,7 +16,7 @@ paddlex.seg.transforms.Compose(transforms)
```python ```python
paddlex.seg.transforms.RandomHorizontalFlip(prob=0.5) paddlex.seg.transforms.RandomHorizontalFlip(prob=0.5)
``` ```
以一定的概率对图像进行水平翻转,模型训练时的数据增强操作。 以一定的概率对图像进行水平翻转模型训练时的数据增强操作。
### 参数 ### 参数
* **prob** (float): 随机水平翻转的概率。默认值为0.5。 * **prob** (float): 随机水平翻转的概率。默认值为0.5。
...@@ -25,7 +25,7 @@ paddlex.seg.transforms.RandomHorizontalFlip(prob=0.5) ...@@ -25,7 +25,7 @@ paddlex.seg.transforms.RandomHorizontalFlip(prob=0.5)
```python ```python
paddlex.seg.transforms.RandomVerticalFlip(prob=0.1) paddlex.seg.transforms.RandomVerticalFlip(prob=0.1)
``` ```
以一定的概率对图像进行垂直翻转,模型训练时的数据增强操作。 以一定的概率对图像进行垂直翻转模型训练时的数据增强操作。
### 参数 ### 参数
* **prob** (float): 随机垂直翻转的概率。默认值为0.1。 * **prob** (float): 随机垂直翻转的概率。默认值为0.1。
...@@ -59,7 +59,7 @@ paddlex.seg.transforms.ResizeByLong(long_size) ...@@ -59,7 +59,7 @@ paddlex.seg.transforms.ResizeByLong(long_size)
```python ```python
paddlex.seg.transforms.ResizeRangeScaling(min_value=400, max_value=600) paddlex.seg.transforms.ResizeRangeScaling(min_value=400, max_value=600)
``` ```
对图像长边随机resize到指定范围内,短边按比例进行缩放,模型训练时的数据增强操作。 对图像长边随机resize到指定范围内,短边按比例进行缩放模型训练时的数据增强操作。
### 参数 ### 参数
* **min_value** (int): 图像长边resize后的最小值。默认值400。 * **min_value** (int): 图像长边resize后的最小值。默认值400。
* **max_value** (int): 图像长边resize后的最大值。默认值600。 * **max_value** (int): 图像长边resize后的最大值。默认值600。
...@@ -124,7 +124,7 @@ paddlex.seg.transforms.RandomBlur(prob=0.1) ...@@ -124,7 +124,7 @@ paddlex.seg.transforms.RandomBlur(prob=0.1)
```python ```python
paddlex.seg.transforms.RandomRotate(rotate_range=15, im_padding_value=[127.5, 127.5, 127.5], label_padding_value=255) paddlex.seg.transforms.RandomRotate(rotate_range=15, im_padding_value=[127.5, 127.5, 127.5], label_padding_value=255)
``` ```
对图像进行随机旋转, 模型训练时的数据增强操作。 对图像进行随机旋转模型训练时的数据增强操作。
在旋转区间[-rotate_range, rotate_range]内,对图像进行随机旋转,当存在标注图像时,同步进行, 在旋转区间[-rotate_range, rotate_range]内,对图像进行随机旋转,当存在标注图像时,同步进行,
并对旋转后的图像和标注图像进行相应的padding。 并对旋转后的图像和标注图像进行相应的padding。
...@@ -138,7 +138,7 @@ paddlex.seg.transforms.RandomRotate(rotate_range=15, im_padding_value=[127.5, 12 ...@@ -138,7 +138,7 @@ paddlex.seg.transforms.RandomRotate(rotate_range=15, im_padding_value=[127.5, 12
```python ```python
paddlex.seg.transforms.RandomScaleAspect(min_scale=0.5, aspect_ratio=0.33) paddlex.seg.transforms.RandomScaleAspect(min_scale=0.5, aspect_ratio=0.33)
``` ```
裁剪并resize回原始尺寸的图像和标注图像,模型训练时的数据增强操作。 裁剪并resize回原始尺寸的图像和标注图像模型训练时的数据增强操作。
按照一定的面积比和宽高比对图像进行裁剪,并reszie回原始图像的图像,当存在标注图时,同步进行。 按照一定的面积比和宽高比对图像进行裁剪,并reszie回原始图像的图像,当存在标注图时,同步进行。
### 参数 ### 参数
......
...@@ -131,7 +131,7 @@ paddlex.transforms.visualize(dataset, ...@@ -131,7 +131,7 @@ paddlex.transforms.visualize(dataset,
对数据预处理/增强中间结果进行可视化。 对数据预处理/增强中间结果进行可视化。
可使用VisualDL查看中间结果: 可使用VisualDL查看中间结果:
1. VisualDL启动方式: visualdl --logdir vdl_output --port 8001 1. VisualDL启动方式: visualdl --logdir vdl_output --port 8001
2. 浏览器打开 https://0.0.0.0:8001即可, 2. 浏览器打开 https://0.0.0.0:8001 即可,
其中0.0.0.0为本机访问,如为远程服务, 改成相应机器IP 其中0.0.0.0为本机访问,如为远程服务, 改成相应机器IP
### 参数 ### 参数
......
...@@ -45,6 +45,7 @@ ...@@ -45,6 +45,7 @@
|[FasterRCNN-ResNet101-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_r101_fpn_1x.tar)| 244.2MB | 119.788 | 38.7 | |[FasterRCNN-ResNet101-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_r101_fpn_1x.tar)| 244.2MB | 119.788 | 38.7 |
|[FasterRCNN-ResNet101_vd-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_r101_vd_fpn_2x.tar) |244.3MB | 156.097 | 40.5 | |[FasterRCNN-ResNet101_vd-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_r101_vd_fpn_2x.tar) |244.3MB | 156.097 | 40.5 |
|[FasterRCNN-HRNet_W18-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_hrnetv2p_w18_1x.tar) |115.5MB | 81.592 | 36 | |[FasterRCNN-HRNet_W18-FPN](https://paddlemodels.bj.bcebos.com/object_detection/faster_rcnn_hrnetv2p_w18_1x.tar) |115.5MB | 81.592 | 36 |
|[PPYOLO](https://paddlemodels.bj.bcebos.com/object_detection/ppyolo_2x.pdparams) | 329.1MB | - |45.9 |
|[YOLOv3-DarkNet53](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_darknet.tar)|249.2MB | 42.672 | 38.9 | |[YOLOv3-DarkNet53](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_darknet.tar)|249.2MB | 42.672 | 38.9 |
|[YOLOv3-MobileNetV1](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1.tar) |99.2MB | 15.442 | 29.3 | |[YOLOv3-MobileNetV1](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v1.tar) |99.2MB | 15.442 | 29.3 |
|[YOLOv3-MobileNetV3_large](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v3.pdparams)|100.7MB | 143.322 | 31.6 | |[YOLOv3-MobileNetV3_large](https://paddlemodels.bj.bcebos.com/object_detection/yolov3_mobilenet_v3.pdparams)|100.7MB | 143.322 | 31.6 |
...@@ -80,6 +81,7 @@ ...@@ -80,6 +81,7 @@
| 模型 | 模型大小 | 预测时间(毫秒) | mIoU(%) | | 模型 | 模型大小 | 预测时间(毫秒) | mIoU(%) |
|:-------|:-----------|:-------------|:----------| |:-------|:-----------|:-------------|:----------|
| [DeepLabv3_MobileNetV3_large_x1_0_ssld](https://paddleseg.bj.bcebos.com/models/deeplabv3p_mobilenetv3_large_cityscapes.tar.gz) | 9.3MB | - | 73.28 |
| [DeepLabv3_MobileNetv2_x1.0](https://paddleseg.bj.bcebos.com/models/mobilenet_cityscapes.tgz) | 14.7MB | - | 69.8 | | [DeepLabv3_MobileNetv2_x1.0](https://paddleseg.bj.bcebos.com/models/mobilenet_cityscapes.tgz) | 14.7MB | - | 69.8 |
| [DeepLabv3_Xception65](https://paddleseg.bj.bcebos.com/models/xception65_bn_cityscapes.tgz) | 329.3MB | - | 79.3 | | [DeepLabv3_Xception65](https://paddleseg.bj.bcebos.com/models/xception65_bn_cityscapes.tgz) | 329.3MB | - | 79.3 |
| [HRNet_W18](https://paddleseg.bj.bcebos.com/models/hrnet_w18_bn_cityscapes.tgz) | 77.3MB | | 79.36 | | [HRNet_W18](https://paddleseg.bj.bcebos.com/models/hrnet_w18_bn_cityscapes.tgz) | 77.3MB | | 79.36 |
......
...@@ -25,12 +25,12 @@ MyDataset/ # 图像分类数据集根目录 ...@@ -25,12 +25,12 @@ MyDataset/ # 图像分类数据集根目录
**为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载图像分类示例数据集](https://bj.bcebos.com/paddlex/datasets/vegetables_cls.tar.gz) **为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载图像分类示例数据集](https://bj.bcebos.com/paddlex/datasets/vegetables_cls.tar.gz)
<!--
> 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中split指标训练集的比例,剩余的比例用于验证集。 > 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中val_value表示验证集的比例,test_value表示测试集的比例(可以为0),剩余的比例用于训练集。
> ``` > ```
> paddlex --split_dataset --from ImageNet --split 0.8 --save_dir ./splited_dataset_dir > paddlex --split_dataset --format ImageNet --dataset_dir MyDataset --val_value 0.2 --test_value 0.1
> ``` > ```
-->
**labels.txt** **labels.txt**
......
...@@ -22,12 +22,10 @@ MyDataset/ # 目标检测数据集根目录 ...@@ -22,12 +22,10 @@ MyDataset/ # 目标检测数据集根目录
**为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载目标检测示例数据集](https://bj.bcebos.com/paddlex/datasets/insect_det.tar.gz) **为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载目标检测示例数据集](https://bj.bcebos.com/paddlex/datasets/insect_det.tar.gz)
<!-- > 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中val_value表示验证集的比例,test_value表示测试集的比例(可以为0),剩余的比例用于训练集。
> 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中split指标训练集的比例,剩余的比例用于验证集。
> ``` > ```
> paddlex --split_dataset --from PascalVOC --pics ./JPEGImages --annotations ./Annotations --split 0.8 --save_dir ./splited_dataset_dir > paddlex --split_dataset --format VOC --dataset_dir MyDataset --val_value 0.2 --test_value 0.1
> ``` > ```
-->
**labels.txt** **labels.txt**
......
...@@ -18,14 +18,12 @@ MyDataset/ # 实例分割数据集根目录 ...@@ -18,14 +18,12 @@ MyDataset/ # 实例分割数据集根目录
在PaddleX中,为了区分训练集和验证集,在`MyDataset`同级目录,使用不同的json表示数据的划分,例如`train.json``val.json`[点击下载实例分割示例数据集](https://bj.bcebos.com/paddlex/datasets/garbage_ins_det.tar.gz) 在PaddleX中,为了区分训练集和验证集,在`MyDataset`同级目录,使用不同的json表示数据的划分,例如`train.json``val.json`[点击下载实例分割示例数据集](https://bj.bcebos.com/paddlex/datasets/garbage_ins_det.tar.gz)
<!-- > 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中val_value表示验证集的比例,test_value表示测试集的比例(可以为0),剩余的比例用于训练集。
> 注:也可使用PaddleX自带工具,对数据集进行随机划分,在数据按照上述示例组织结构后,使用如下命令,即可快速完成数据集随机划分,其中split指定训练集的比例,剩余比例用于验证集。
> ``` > ```
> paddlex --split_dataset --from MSCOCO --pics ./JPEGImages --annotations ./annotations.json --split 0.8 --save_dir ./splited_dataset_dir > paddlex --split_dataset --format COCO --dataset_dir MyDataset --val_value 0.2 --test_value 0.1
> ``` > ```
-->
MSCOCO数据的标注文件采用json格式,用户可使用Labelme, 精灵标注助手或EasyData等标注工具进行标注,参见[数据标注工具](../annotations.md) MSCOCO数据的标注文件采用json格式,用户可使用Labelme, 精灵标注助手或EasyData等标注工具进行标注,参见[数据标注工具](../annotation.md)
## PaddleX加载数据集 ## PaddleX加载数据集
示例代码如下, 示例代码如下,
......
...@@ -23,12 +23,10 @@ MyDataset/ # 语义分割数据集根目录 ...@@ -23,12 +23,10 @@ MyDataset/ # 语义分割数据集根目录
**为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载语义分割示例数据集](https://bj.bcebos.com/paddlex/datasets/optic_disc_seg.tar.gz) **为了用于训练,我们需要在`MyDataset`目录下准备`train_list.txt`, `val_list.txt`和`labels.txt`三个文件**,分别用于表示训练集列表,验证集列表和类别标签列表。[点击下载语义分割示例数据集](https://bj.bcebos.com/paddlex/datasets/optic_disc_seg.tar.gz)
<!-- > 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中val_value表示验证集的比例,test_value表示测试集的比例(可以为0),剩余的比例用于训练集。
> 注:也可使用PaddleX自带工具,对数据集进行随机划分,**在数据集按照上面格式组织后**,使用如下命令即可快速完成数据集随机划分,其中split指标训练集的比例,剩余的比例用于验证集。
> ``` > ```
> paddlex --split_dataset --from Seg --pics ./JPEGImages --annotations ./Annotations --split 0.8 --save_dir ./splited_dataset_dir > paddlex --split_dataset --format Seg --dataset_dir MyDataset --val_value 0.2 --test_value 0.1
> ``` > ```
-->
**labels.txt** **labels.txt**
......
# 轻量级服务化部署
## 简介
借助`PaddleHub-Serving`,可以将`PaddleX``Inference Model`进行快速部署,以提供在线预测的能力。
关于`PaddleHub-Serving`的更多信息,可参照[PaddleHub-Serving](https://github.com/PaddlePaddle/PaddleHub/blob/develop/docs/tutorial/serving.md)
**注意:使用此方式部署,需确保自己Python环境中PaddleHub的版本高于1.8.0, 可在命令终端输入`pip show paddlehub`确认版本信息。**
下面,我们按照步骤,实现将一个图像分类模型[MobileNetV3_small_ssld](https://bj.bcebos.com/paddlex/models/mobilenetv3_small_ssld_imagenet.tar.gz)转换成`PaddleHub`的预训练模型,并利用`PaddleHub-Serving`实现一键部署。
# 模型部署
## 1 部署模型准备
部署模型的格式均为目录下包含`__model__``__params__``model.yml`三个文件,如若不然,则参照[部署模型导出文档](./export_model.md)进行导出。
## 2 模型转换
首先,我们将`PaddleX``Inference Model`转换成`PaddleHub`的预训练模型,使用命令`hub convert`即可一键转换,对此命令的说明如下:
```shell
$ hub convert --model_dir XXXX \
--module_name XXXX \
--module_version XXXX \
--output_dir XXXX
```
**参数**
|参数|用途|
|-|-|
|--model_dir/-m|`PaddleX Inference Model`所在的目录|
|--module_name/-n|生成预训练模型的名称|
|--module_version/-v|生成预训练模型的版本,默认为`1.0.0`|
|--output_dir/-o|生成预训练模型的存放位置,默认为`{module_name}_{timestamp}`|
因此,我们仅需要一行命令即可完成预训练模型的转换。
```shell
hub convert --model_dir mobilenetv3_small_ssld_imagenet_hub --module_name mobilenetv3_small_ssld_imagenet_hub
```
转换成功后会打印提示信息,如下:
```shell
$ The converted module is stored in `MobileNetV3_small_ssld_hub_1596077881.868501`.
```
等待生成成功的提示后,我们就在输出目录中得到了一个`PaddleHub`的一个预训练模型。
## 3 模型安装
在模型转换一步中,我们得到了一个`.tar.gz`格式的预训练模型压缩包,在进行部署之前需要先安装到本机,使用命令`hub install`即可一键安装,对此命令的说明如下:
```shell
$ hub install ${MODULE}
```
其中${MODULE}为要安装的预训练模型文件路径。
因此,我们使用`hub install`命令安装:
```shell
hub install MobileNetV3_small_ssld_hub_1596077881.868501/mobilenetv3_small_ssld_imagenet_hub.tar.gz
```
安装成功后会打印提示信息,如下:
```shell
$ Successfully installed mobilenetv3_small_ssld_imagenet_hub
```
## 4 模型部署
下面,我们只需要使用`hub serving`命令即可完成模型的一键部署,对此命令的说明如下:
```shell
$ hub serving start --modules/-m [Module1==Version1, Module2==Version2, ...] \
--port/-p XXXX
--config/-c XXXX
```
**参数**
|参数|用途|
|-|-|
|--modules/-m|PaddleHub Serving预安装模型,以多个Module==Version键值对的形式列出<br>*`当不指定Version时,默认选择最新版本`*|
|--port/-p|服务端口,默认为8866|
|--config/-c|使用配置文件配置模型|
因此,我们仅需要一行代码即可完成模型的部署,如下:
```shell
$ hub serving start -m mobilenetv3_small_ssld_imagenet_hub
```
等待模型加载后,此预训练模型就已经部署在机器上了。
我们还可以使用配置文件对部署的模型进行更多配置,配置文件格式如下:
```json
{
"modules_info": {
"mobilenetv3_small_ssld_imagenet_hub": {
"init_args": {
"version": "1.0.0"
},
"predict_args": {
"batch_size": 1,
"use_gpu": false
}
}
},
"port": 8866
}
```
|参数|用途|
|-|-|
|modules_info|PaddleHub Serving预安装模型,以字典列表形式列出,key为模型名称。其中:<br>`init_args`为模型加载时输入的参数,等同于`paddlehub.Module(**init_args)`<br>`predict_args`为模型预测时输入的参数,以`mobilenetv3_small_ssld_imagenet_hub`为例,等同于`mobilenetv3_small_ssld_imagenet_hub.batch_predict(**predict_args)`
|port|服务端口,默认为8866|
## 5 测试
在第二步模型安装的同时,会生成一个客户端请求示例,存放在模型安装目录,默认为`${HUB_HOME}/.paddlehub/modules`,对于此例,我们可以在`~/.paddlehub/modules/mobilenetv3_small_ssld_imagenet_hub`找到此客户端示例`serving_client_demo.py`,代码如下:
```python
# coding: utf8
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
if __name__ == '__main__':
# 获取图片的base64编码格式
img1 = cv2_to_base64(cv2.imread("IMAGE_PATH1"))
img2 = cv2_to_base64(cv2.imread("IMAGE_PATH2"))
data = {'images': [img1, img2]}
# 指定content-type
headers = {"Content-type": "application/json"}
# 发送HTTP请求
url = "http://127.0.0.1:8866/predict/mobilenetv3_small_ssld_imagenet_hub"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 打印预测结果
print(r.json()["results"])
```
使用的测试图片如下:
![](../train/images/test.jpg)
将代码中的`IMAGE_PATH1`改成想要进行预测的图片路径后,在命令行执行:
```python
python ~/.paddlehub/module/MobileNetV3_small_ssld_hub/serving_client_demo.py
```
即可收到预测结果,如下:
```shell
[[{'category': 'envelope', 'category_id': 549, 'score': 0.2141510397195816}]]
````
到此,我们就完成了`PaddleX`模型的一键部署。
...@@ -7,6 +7,7 @@ ...@@ -7,6 +7,7 @@
:caption: 文档目录: :caption: 文档目录:
export_model.md export_model.md
hub_serving.md
server/index server/index
nvidia-jetson.md nvidia-jetson.md
paddlelite/index paddlelite/index
# Nvidia Jetson开发板 # Nvidia Jetson开发板
## 说明 ## 说明
本文档在 `Linux`平台使用`GCC 7.4`测试过,如果需要使用更高G++版本编译使用,则需要重新编译Paddle预测库,请参考: [Nvidia Jetson嵌入式硬件预测库源码编译](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/advanced_guide/inference_deployment/inference/build_and_install_lib_cn.html#id12) 本文档在基于Nvidia Jetpack 4.4的`Linux`平台上使用`GCC 7.4`测试过,如需使用不同G++版本,则需要重新编译Paddle预测库,请参考: [NVIDIA Jetson嵌入式硬件预测库源码编译](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/advanced_guide/inference_deployment/inference/build_and_install_lib_cn.html#id12)
## 前置条件 ## 前置条件
* G++ 7.4 * G++ 7.4
* CUDA 9.0 / CUDA 10.0, CUDNN 7+ (仅在使用GPU版本的预测库时需要) * CUDA 10.0 / CUDNN 8 (仅在使用GPU版本的预测库时需要)
* CMake 3.0+ * CMake 3.0+
请确保系统已经安装好上述基本软件,**下面所有示例以工作目录 `/root/projects/`演示** 请确保系统已经安装好上述基本软件,**下面所有示例以工作目录 `/root/projects/`演示**
...@@ -57,13 +57,6 @@ CUDA_LIB=/usr/local/cuda/lib64 ...@@ -57,13 +57,6 @@ CUDA_LIB=/usr/local/cuda/lib64
# CUDNN 的 lib 路径 # CUDNN 的 lib 路径
CUDNN_LIB=/usr/local/cuda/lib64 CUDNN_LIB=/usr/local/cuda/lib64
# 是否加载加密后的模型
WITH_ENCRYPTION=OFF
# OPENCV 路径, 如果使用自带预编译版本可不修改
sh $(pwd)/scripts/jetson_bootstrap.sh # 下载预编译版本的opencv
OPENCV_DIR=$(pwd)/deps/opencv3/
# 以下无需改动 # 以下无需改动
rm -rf build rm -rf build
mkdir -p build mkdir -p build
...@@ -77,18 +70,13 @@ cmake .. \ ...@@ -77,18 +70,13 @@ cmake .. \
-DPADDLE_DIR=${PADDLE_DIR} \ -DPADDLE_DIR=${PADDLE_DIR} \
-DWITH_STATIC_LIB=${WITH_STATIC_LIB} \ -DWITH_STATIC_LIB=${WITH_STATIC_LIB} \
-DCUDA_LIB=${CUDA_LIB} \ -DCUDA_LIB=${CUDA_LIB} \
-DCUDNN_LIB=${CUDNN_LIB} \ -DCUDNN_LIB=${CUDNN_LIB}
-DENCRYPTION_DIR=${ENCRYPTION_DIR} \
-DOPENCV_DIR=${OPENCV_DIR}
make make
``` ```
**注意:** linux环境下编译会自动下载OPENCV和YAML,如果编译环境无法访问外网,可手动下载: **注意:** linux环境下编译会自动下载YAML,如果编译环境无法访问外网,可手动下载:
- [opencv3_aarch.tgz](https://bj.bcebos.com/paddlex/deploy/tools/opencv3_aarch.tgz)
- [yaml-cpp.zip](https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip) - [yaml-cpp.zip](https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip)
opencv3_aarch.tgz文件下载后解压,然后在script/build.sh中指定`OPENCE_DIR`为解压后的路径。
yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip` 中的网址,改为下载文件的路径。 yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://bj.bcebos.com/paddlex/deploy/deps/yaml-cpp.zip` 中的网址,改为下载文件的路径。
修改脚本设置好主要参数后,执行`build`脚本: 修改脚本设置好主要参数后,执行`build`脚本:
...@@ -100,7 +88,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -100,7 +88,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
**在加载模型前,请检查你的模型目录中文件应该包括`model.yml`、`__model__`和`__params__`三个文件。如若不满足这个条件,请参考[模型导出为Inference文档](export_model.md)将模型导出为部署格式。** **在加载模型前,请检查你的模型目录中文件应该包括`model.yml`、`__model__`和`__params__`三个文件。如若不满足这个条件,请参考[模型导出为Inference文档](export_model.md)将模型导出为部署格式。**
编译成功后,预测demo的可执行程序分别为`build/demo/detector``build/demo/classifier``build/demo/segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下: * 编译成功后,图片预测demo的可执行程序分别为`build/demo/detector``build/demo/classifier``build/demo/segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 | | 参数 | 说明 |
| ---- | ---- | | ---- | ---- |
...@@ -111,10 +99,26 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -111,10 +99,26 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
| use_trt | 是否使用 TensorRT 预测, 支持值为0或1(默认值为0) | | use_trt | 是否使用 TensorRT 预测, 支持值为0或1(默认值为0) |
| gpu_id | GPU 设备ID, 默认值为0 | | gpu_id | GPU 设备ID, 默认值为0 |
| save_dir | 保存可视化结果的路径, 默认值为"output",**classfier无该参数** | | save_dir | 保存可视化结果的路径, 默认值为"output",**classfier无该参数** |
| key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 |
| batch_size | 预测的批量大小,默认为1 | | batch_size | 预测的批量大小,默认为1 |
| thread_num | 预测的线程数,默认为cpu处理器个数 | | thread_num | 预测的线程数,默认为cpu处理器个数 |
| use_ir_optim | 是否使用图优化策略,支持值为0或1(默认值为1,图像分割默认值为0)|
* 编译成功后,视频预测demo的可执行程序分别为`build/demo/video_detector``build/demo/video_classifier``build/demo/video_segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 |
| ---- | ---- |
| model_dir | 导出的预测模型所在路径 |
| use_camera | 是否使用摄像头预测,支持值为0或1(默认值为0) |
| camera_id | 摄像头设备ID,默认值为0 |
| video_path | 视频文件的路径 |
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0) |
| use_trt | 是否使用 TensorRT 预测, 支持值为0或1(默认值为0) |
| gpu_id | GPU 设备ID, 默认值为0 |
| show_result | 对视频文件做预测时,是否在屏幕上实时显示预测可视化结果(因加入了延迟处理,故显示结果不能反映真实的帧率),支持值为0或1(默认值为0) |
| save_result | 是否将每帧的预测可视结果保存为视频文件,支持值为0或1(默认值为1) |
| save_dir | 保存可视化结果的路径, 默认值为"output" |
**注意:若系统无GUI,则不要将show_result设置为1。当使用摄像头预测时,按`ESC`键可关闭摄像头并推出预测程序。**
## 样例 ## 样例
...@@ -143,3 +147,21 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -143,3 +147,21 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
./build/demo/detector --model_dir=/root/projects/inference_model --image_list=/root/projects/images_list.txt --use_gpu=1 --save_dir=output --batch_size=2 --thread_num=2 ./build/demo/detector --model_dir=/root/projects/inference_model --image_list=/root/projects/images_list.txt --use_gpu=1 --save_dir=output --batch_size=2 --thread_num=2
``` ```
图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。 图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。
**样例三:**
使用摄像头预测:
```shell
./build/demo/video_detector --model_dir=/root/projects/inference_model --use_camera=1 --use_gpu=1 --save_dir=output --save_result=1
```
`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。
**样例四:**
对视频文件进行预测:
```shell
./build/demo/video_detector --model_dir=/root/projects/inference_model --video_path=/path/to/video_file --use_gpu=1 --save_dir=output --show_result=1 --save_result=1
```
`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。如果系统有GUI,通过将`show_result`设置为1在屏幕上观看可视化预测结果。
...@@ -49,7 +49,7 @@ PaddleX提供了两种方式: ...@@ -49,7 +49,7 @@ PaddleX提供了两种方式:
### 语义分割 ### 语义分割
实验背景:使用UNet模型,数据集为视盘分割示例数据,剪裁训练代码见[tutorials/compress/segmentation](https://github.com/PaddlePaddle/PaddleX/tree/develop/tutorials/compress/segmentation) 实验背景:使用UNet模型,数据集为视盘分割示例数据,剪裁训练代码见[tutorials/compress/segmentation](https://github.com/PaddlePaddle/PaddleX/tree/develop/tutorials/compress/segmentation)
| 模型 | 剪裁情况 | 模型大小 | mIOU(%) |GPU预测速度 | CPU预测速度 | | 模型 | 剪裁情况 | 模型大小 | mIoU(%) |GPU预测速度 | CPU预测速度 |
| :-----| :--------| :-------- | :---------- |:---------- | :---------| | :-----| :--------| :-------- | :---------- |:---------- | :---------|
|UNet | 无剪裁(原模型)| 77M | 91.22 |33.28ms |9523.55ms | |UNet | 无剪裁(原模型)| 77M | 91.22 |33.28ms |9523.55ms |
|UNet | 方案一(eval_metric_loss=0.10) |26M | 90.37 |21.04ms |3936.20ms | |UNet | 方案一(eval_metric_loss=0.10) |26M | 90.37 |21.04ms |3936.20ms |
......
...@@ -6,7 +6,7 @@ ...@@ -6,7 +6,7 @@
定点量化使用更少的比特数(如8-bit、3-bit、2-bit等)表示神经网络的权重和激活值,从而加速模型推理速度。PaddleX提供了训练后量化技术,其原理可参见[训练后量化原理](https://paddlepaddle.github.io/PaddleSlim/algo/algo.html#id14),该量化使用KL散度确定量化比例因子,将FP32模型转成INT8模型,且不需要重新训练,可以快速得到量化模型。 定点量化使用更少的比特数(如8-bit、3-bit、2-bit等)表示神经网络的权重和激活值,从而加速模型推理速度。PaddleX提供了训练后量化技术,其原理可参见[训练后量化原理](https://paddlepaddle.github.io/PaddleSlim/algo/algo.html#id14),该量化使用KL散度确定量化比例因子,将FP32模型转成INT8模型,且不需要重新训练,可以快速得到量化模型。
## 使用PaddleX量化模型 ## 使用PaddleX量化模型
PaddleX提供了`export_quant_model`接口,让用户以接口的形式对训练后的模型进行量化。点击查看[量化接口使用文档](../../../apis/slim.html) PaddleX提供了`export_quant_model`接口,让用户以接口的形式对训练后的模型进行量化。点击查看[量化接口使用文档](../../../apis/slim.md)
## 量化性能对比 ## 量化性能对比
模型量化后的性能对比指标请查阅[PaddleSlim模型库](https://paddlepaddle.github.io/PaddleSlim/model_zoo.html) 模型量化后的性能对比指标请查阅[PaddleSlim模型库](https://paddlepaddle.github.io/PaddleSlim/model_zoo.html)
...@@ -116,7 +116,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -116,7 +116,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
**在加载模型前,请检查你的模型目录中文件应该包括`model.yml`、`__model__`和`__params__`三个文件。如若不满足这个条件,请参考[模型导出为Inference文档](../../export_model.md)将模型导出为部署格式。** **在加载模型前,请检查你的模型目录中文件应该包括`model.yml`、`__model__`和`__params__`三个文件。如若不满足这个条件,请参考[模型导出为Inference文档](../../export_model.md)将模型导出为部署格式。**
编译成功后,预测demo的可执行程序分别为`build/demo/detector``build/demo/classifier``build/demo/segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下: * 编译成功后,图片预测demo的可执行程序分别为`build/demo/detector``build/demo/classifier``build/demo/segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 | | 参数 | 说明 |
| ---- | ---- | | ---- | ---- |
...@@ -130,7 +130,24 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -130,7 +130,24 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
| key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 | | key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 |
| batch_size | 预测的批量大小,默认为1 | | batch_size | 预测的批量大小,默认为1 |
| thread_num | 预测的线程数,默认为cpu处理器个数 | | thread_num | 预测的线程数,默认为cpu处理器个数 |
| use_ir_optim | 是否使用图优化策略,支持值为0或1(默认值为1,图像分割默认值为0)|
* 编译成功后,视频预测demo的可执行程序分别为`build/demo/video_detector``build/demo/video_classifier``build/demo/video_segmenter`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 |
| ---- | ---- |
| model_dir | 导出的预测模型所在路径 |
| use_camera | 是否使用摄像头预测,支持值为0或1(默认值为0) |
| camera_id | 摄像头设备ID,默认值为0 |
| video_path | 视频文件的路径 |
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0) |
| use_trt | 是否使用 TensorRT 预测, 支持值为0或1(默认值为0) |
| gpu_id | GPU 设备ID, 默认值为0 |
| show_result | 对视频文件做预测时,是否在屏幕上实时显示预测可视化结果(因加入了延迟处理,故显示结果不能反映真实的帧率),支持值为0或1(默认值为0) |
| save_result | 是否将每帧的预测可视结果保存为视频文件,支持值为0或1(默认值为1) |
| save_dir | 保存可视化结果的路径, 默认值为"output"|
| key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 |
**注意:若系统无GUI,则不要将show_result设置为1。当使用摄像头预测时,按`ESC`键可关闭摄像头并推出预测程序。**
## 样例 ## 样例
...@@ -138,7 +155,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -138,7 +155,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
> 关于预测速度的说明:加载模型后前几张图片的预测速度会较慢,这是因为运行启动时涉及到内存显存初始化等步骤,通常在预测20-30张图片后模型的预测速度达到稳定。 > 关于预测速度的说明:加载模型后前几张图片的预测速度会较慢,这是因为运行启动时涉及到内存显存初始化等步骤,通常在预测20-30张图片后模型的预测速度达到稳定。
`样例一` **样例一:**
不使用`GPU`测试图片 `/root/projects/images/xiaoduxiong.jpeg` 不使用`GPU`测试图片 `/root/projects/images/xiaoduxiong.jpeg`
...@@ -148,7 +165,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -148,7 +165,7 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。 图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。
`样例二`: **样例二:**
使用`GPU`预测多个图片`/root/projects/image_list.txt`,image_list.txt内容的格式如下: 使用`GPU`预测多个图片`/root/projects/image_list.txt`,image_list.txt内容的格式如下:
``` ```
...@@ -161,3 +178,21 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https:// ...@@ -161,3 +178,21 @@ yaml-cpp.zip文件下载后无需解压,在cmake/yaml.cmake中将`URL https://
./build/demo/detector --model_dir=/root/projects/inference_model --image_list=/root/projects/images_list.txt --use_gpu=1 --save_dir=output --batch_size=2 --thread_num=2 ./build/demo/detector --model_dir=/root/projects/inference_model --image_list=/root/projects/images_list.txt --use_gpu=1 --save_dir=output --batch_size=2 --thread_num=2
``` ```
图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。 图片文件`可视化预测结果`会保存在`save_dir`参数设置的目录下。
**样例三:**
使用摄像头预测:
```shell
./build/demo/video_detector --model_dir=/root/projects/inference_model --use_camera=1 --use_gpu=1 --save_dir=output --save_result=1
```
`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。
**样例四:**
对视频文件进行预测:
```shell
./build/demo/video_detector --model_dir=/root/projects/inference_model --video_path=/path/to/video_file --use_gpu=1 --save_dir=output --show_result=1 --save_result=1
```
`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。如果系统有GUI,通过将`show_result`设置为1在屏幕上观看可视化预测结果。
...@@ -101,7 +101,7 @@ D: ...@@ -101,7 +101,7 @@ D:
cd D:\projects\PaddleX\deploy\cpp\out\build\x64-Release cd D:\projects\PaddleX\deploy\cpp\out\build\x64-Release
``` ```
编译成功后,预测demo的入口程序为`paddlex_inference\detector.exe`,`paddlex_inference\classifier.exe`,`paddlex_inference\segmenter.exe`,用户可根据自己的模型类型选择,其主要命令参数说明如下: * 编译成功后,图片预测demo的入口程序为`paddlex_inference\detector.exe`,`paddlex_inference\classifier.exe`,`paddlex_inference\segmenter.exe`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 | | 参数 | 说明 |
| ---- | ---- | | ---- | ---- |
...@@ -114,7 +114,24 @@ cd D:\projects\PaddleX\deploy\cpp\out\build\x64-Release ...@@ -114,7 +114,24 @@ cd D:\projects\PaddleX\deploy\cpp\out\build\x64-Release
| key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 | | key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 |
| batch_size | 预测的批量大小,默认为1 | | batch_size | 预测的批量大小,默认为1 |
| thread_num | 预测的线程数,默认为cpu处理器个数 | | thread_num | 预测的线程数,默认为cpu处理器个数 |
| use_ir_optim | 是否使用图优化策略,支持值为0或1(默认值为1,图像分割默认值为0)|
* 编译成功后,视频预测demo的入口程序为`paddlex_inference\video_detector.exe`,`paddlex_inference\video_classifier.exe`,`paddlex_inference\video_segmenter.exe`,用户可根据自己的模型类型选择,其主要命令参数说明如下:
| 参数 | 说明 |
| ---- | ---- |
| model_dir | 导出的预测模型所在路径 |
| use_camera | 是否使用摄像头预测,支持值为0或1(默认值为0) |
| camera_id | 摄像头设备ID,默认值为0 |
| video_path | 视频文件的路径 |
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0) |
| gpu_id | GPU 设备ID, 默认值为0 |
| show_result | 对视频文件做预测时,是否在屏幕上实时显示预测可视化结果(因加入了延迟处理,故显示结果不能反映真实的帧率),支持值为0或1(默认值为0) |
| save_result | 是否将每帧的预测可视结果保存为视频文件,支持值为0或1(默认值为1) |
| save_dir | 保存可视化结果的路径, 默认值为"output" |
| key | 加密过程中产生的密钥信息,默认值为""表示加载的是未加密的模型 |
**注意:若系统无GUI,则不要将show_result设置为1。当使用摄像头预测时,按`ESC`键可关闭摄像头并推出预测程序。**
## 样例 ## 样例
...@@ -157,3 +174,18 @@ D:\images\xiaoduxiongn.jpeg ...@@ -157,3 +174,18 @@ D:\images\xiaoduxiongn.jpeg
``` ```
`--key`传入加密工具输出的密钥,例如`kLAl1qOs5uRbFt0/RrIDTZW2+tOf5bzvUIaHGF8lJ1c=`, 图片文件可视化预测结果会保存在`save_dir`参数设置的目录下。 `--key`传入加密工具输出的密钥,例如`kLAl1qOs5uRbFt0/RrIDTZW2+tOf5bzvUIaHGF8lJ1c=`, 图片文件可视化预测结果会保存在`save_dir`参数设置的目录下。
### 样例四:(使用未加密的模型开启摄像头预测)
```shell
.\paddlex_inference\video_detector.exe --model_dir=D:\projects\inference_model --use_camera=1 --use_gpu=1 --save_dir=output
```
当`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。
### 样例五:(使用未加密的模型对视频文件做预测)
```shell
.\paddlex_inference\video_detector.exe --model_dir=D:\projects\inference_model --video_path=D:\projects\video_test.mp4 --use_gpu=1 --show_result=1 --save_dir=output
```
当`save_result`设置为1时,`可视化预测结果`会以视频文件的格式保存在`save_dir`参数设置的目录下。如果系统有GUI,通过将`show_result`设置为1在屏幕上观看可视化预测结果。
...@@ -51,7 +51,7 @@ paddlex-encryption ...@@ -51,7 +51,7 @@ paddlex-encryption
| |
├── lib # libpmodel-encrypt.so和libpmodel-decrypt.so动态库 ├── lib # libpmodel-encrypt.so和libpmodel-decrypt.so动态库
| |
└── tool # paddlex_encrypt_tool └── tool # paddle_encrypt_tool
``` ```
Windows加密工具包含内容为: Windows加密工具包含内容为:
...@@ -61,7 +61,7 @@ paddlex-encryption ...@@ -61,7 +61,7 @@ paddlex-encryption
| |
├── lib # pmodel-encrypt.dll和pmodel-decrypt.dll动态库 pmodel-encrypt.lib和pmodel-encrypt.lib静态库 ├── lib # pmodel-encrypt.dll和pmodel-decrypt.dll动态库 pmodel-encrypt.lib和pmodel-encrypt.lib静态库
| |
└── tool # paddlex_encrypt_tool.exe 模型加密工具 └── tool # paddle_encrypt_tool.exe 模型加密工具
``` ```
### 1.3 加密PaddleX模型 ### 1.3 加密PaddleX模型
...@@ -71,13 +71,13 @@ paddlex-encryption ...@@ -71,13 +71,13 @@ paddlex-encryption
Linux平台: Linux平台:
``` ```
# 假设模型在/root/projects下 # 假设模型在/root/projects下
./paddlex-encryption/tool/paddlex_encrypt_tool -model_dir /root/projects/paddlex_inference_model -save_dir /root/projects/paddlex_encrypted_model ./paddlex-encryption/tool/paddle_encrypt_tool -model_dir /root/projects/paddlex_inference_model -save_dir /root/projects/paddlex_encrypted_model
``` ```
Windows平台: Windows平台:
``` ```
# 假设模型在D:/projects下 # 假设模型在D:/projects下
.\paddlex-encryption\tool\paddlex_encrypt_tool.exe -model_dir D:\projects\paddlex_inference_model -save_dir D:\projects\paddlex_encrypted_model .\paddlex-encryption\tool\paddle_encrypt_tool.exe -model_dir D:\projects\paddlex_inference_model -save_dir D:\projects\paddlex_encrypted_model
``` ```
`-model_dir`用于指定inference模型路径(参考[导出inference模型](../export_model.md)将模型导出为inference格式模型),可使用[导出小度熊识别模型](../export_model.md)中导出的`inference_model`。加密完成后,加密过的模型会保存至指定的`-save_dir`下,包含`__model__.encrypted``__params__.encrypted``model.yml`三个文件,同时生成密钥信息,命令输出如下图所示,密钥为`kLAl1qOs5uRbFt0/RrIDTZW2+tOf5bzvUIaHGF8lJ1c=` `-model_dir`用于指定inference模型路径(参考[导出inference模型](../export_model.md)将模型导出为inference格式模型),可使用[导出小度熊识别模型](../export_model.md)中导出的`inference_model`。加密完成后,加密过的模型会保存至指定的`-save_dir`下,包含`__model__.encrypted``__params__.encrypted``model.yml`三个文件,同时生成密钥信息,命令输出如下图所示,密钥为`kLAl1qOs5uRbFt0/RrIDTZW2+tOf5bzvUIaHGF8lJ1c=`
......
...@@ -27,7 +27,26 @@ import paddlex as pdx ...@@ -27,7 +27,26 @@ import paddlex as pdx
predictor = pdx.deploy.Predictor('./inference_model') predictor = pdx.deploy.Predictor('./inference_model')
image_list = ['xiaoduxiong_test_image/JPEGImages/WeChatIMG110.jpeg', image_list = ['xiaoduxiong_test_image/JPEGImages/WeChatIMG110.jpeg',
'xiaoduxiong_test_image/JPEGImages/WeChatIMG111.jpeg'] 'xiaoduxiong_test_image/JPEGImages/WeChatIMG111.jpeg']
result = predictor.predict(image_list=image_list) result = predictor.batch_predict(image_list=image_list)
```
* 视频流预测
```
import cv2
import paddlex as pdx
predictor = pdx.deploy.Predictor('./inference_model')
cap = cv2.VideoCapture(0)
while cap.isOpened():
ret, frame = cap.read()
if ret:
result = predictor.predict(frame)
vis_img = pdx.det.visualize(frame, result, threshold=0.6, save_dir=None)
cv2.imshow('Xiaoduxiong', vis_img)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap.release()
``` ```
> 关于预测速度的说明:加载模型后前几张图片的预测速度会较慢,这是因为运行启动时涉及到内存显存初始化等步骤,通常在预测20-30张图片后模型的预测速度达到稳定。 > 关于预测速度的说明:加载模型后前几张图片的预测速度会较慢,这是因为运行启动时涉及到内存显存初始化等步骤,通常在预测20-30张图片后模型的预测速度达到稳定。
......
# 人像分割模型 # 人像分割模型
本教程基于PaddleX核心分割模型实现人像分割,开放预训练模型和测试数据、支持视频流人像分割、提供模型Fine-tune到Paddle Lite移动端部署的全流程应用指南。 本教程基于PaddleX核心分割模型实现人像分割,开放预训练模型和测试数据、支持视频流人像分割、提供模型Fine-tune到Paddle Lite移动端及Nvidia Jeston嵌入式设备部署的全流程应用指南。
## 预训练模型和测试数据 ## 预训练模型和测试数据
#### 预训练模型 #### 预训练模型
本案例开放了两个在大规模人像数据集上训练好的模型,以满足服务器端场景和移动端场景的需求。使用这些模型可以快速体验视频流人像分割,也可以部署到移动端进行实时人像分割,也可以用于完成模型Fine-tuning。 本案例开放了两个在大规模人像数据集上训练好的模型,以满足服务器端场景和移动端场景的需求。使用这些模型可以快速体验视频流人像分割,也可以部署到移动端或嵌入式设备进行实时人像分割,也可以用于完成模型Fine-tuning。
| 模型类型 | Checkpoint Parameter | Inference Model | Quant Inference Model | 备注 | | 模型类型 | Checkpoint Parameter | Inference Model | Quant Inference Model | 备注 |
| --- | --- | --- | ---| --- | | --- | --- | --- | ---| --- |
...@@ -243,15 +243,17 @@ python quant_offline.py --model_dir output/best_model \ ...@@ -243,15 +243,17 @@ python quant_offline.py --model_dir output/best_model \
* `--save_dir`: 量化模型保存路径 * `--save_dir`: 量化模型保存路径
* `--image_shape`: 网络输入图像大小(w, h) * `--image_shape`: 网络输入图像大小(w, h)
## Paddle Lite移动端部署 ## 推理部署
### Paddle Lite移动端部署
本案例将人像分割模型在移动端进行部署,部署流程展示如下,通用的移动端部署流程参见[Paddle Lite移动端部署](../../docs/deploy/paddlelite/android.md) 本案例将人像分割模型在移动端进行部署,部署流程展示如下,通用的移动端部署流程参见[Paddle Lite移动端部署](../../docs/deploy/paddlelite/android.md)
### 1. 将PaddleX模型导出为inference模型 #### 1. 将PaddleX模型导出为inference模型
本案例使用humanseg_mobile_quant预训练模型,该模型已经是inference模型,不需要再执行模型导出步骤。如果不使用预训练模型,则执行上一章节`模型训练`中的`模型导出`将自己训练的模型导出为inference格式。 本案例使用humanseg_mobile_quant预训练模型,该模型已经是inference模型,不需要再执行模型导出步骤。如果不使用预训练模型,则执行上一章节`模型训练`中的`模型导出`将自己训练的模型导出为inference格式。
### 2. 将inference模型优化为Paddle Lite模型 #### 2. 将inference模型优化为Paddle Lite模型
下载并解压 [模型优化工具opt](https://bj.bcebos.com/paddlex/deploy/lite/model_optimize_tool_11cbd50e.tar.gz),进入模型优化工具opt所在路径后,执行以下命令: 下载并解压 [模型优化工具opt](https://bj.bcebos.com/paddlex/deploy/lite/model_optimize_tool_11cbd50e.tar.gz),进入模型优化工具opt所在路径后,执行以下命令:
...@@ -273,16 +275,16 @@ python quant_offline.py --model_dir output/best_model \ ...@@ -273,16 +275,16 @@ python quant_offline.py --model_dir output/best_model \
更详细的使用方法和参数含义请参考: [使用opt转化模型](https://paddle-lite.readthedocs.io/zh/latest/user_guides/opt/opt_bin.html) 更详细的使用方法和参数含义请参考: [使用opt转化模型](https://paddle-lite.readthedocs.io/zh/latest/user_guides/opt/opt_bin.html)
### 3. 移动端预测 #### 3. 移动端预测
PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图像分类、目标检测、实例分割和语义分割,该demo位于`PaddleX/deploy/lite/android/demo`,用户将模型、配置文件和测试图片拷贝至该demo下进行预测。 PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图像分类、目标检测、实例分割和语义分割,该demo位于`PaddleX/deploy/lite/android/demo`,用户将模型、配置文件和测试图片拷贝至该demo下进行预测。
#### 3.1 前置依赖 ##### 3.1 前置依赖
* Android Studio 3.4 * Android Studio 3.4
* Android手机或开发板 * Android手机或开发板
#### 3.2 拷贝模型、配置文件和测试图片 ##### 3.2 拷贝模型、配置文件和测试图片
* 将Lite模型(.nb文件)拷贝到`PaddleX/deploy/lite/android/demo/app/src/main/assets/model/`目录下, 根据.nb文件的名字,修改文件`PaddleX/deploy/lite/android/demo/app/src/main/res/values/strings.xml`中的`MODEL_PATH_DEFAULT` * 将Lite模型(.nb文件)拷贝到`PaddleX/deploy/lite/android/demo/app/src/main/assets/model/`目录下, 根据.nb文件的名字,修改文件`PaddleX/deploy/lite/android/demo/app/src/main/res/values/strings.xml`中的`MODEL_PATH_DEFAULT`
...@@ -290,7 +292,7 @@ PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图 ...@@ -290,7 +292,7 @@ PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图
* 将测试图片拷贝到`PaddleX/deploy/lite/android/demo/app/src/main/assets/images/`目录下,根据图片文件的名字,修改文件`PaddleX/deploy/lite/android/demo/app/src/main/res/values/strings.xml`中的`IMAGE_PATH_DEFAULT` * 将测试图片拷贝到`PaddleX/deploy/lite/android/demo/app/src/main/assets/images/`目录下,根据图片文件的名字,修改文件`PaddleX/deploy/lite/android/demo/app/src/main/res/values/strings.xml`中的`IMAGE_PATH_DEFAULT`
#### 3.3 导入工程并运行 ##### 3.3 导入工程并运行
* 打开Android Studio,在"Welcome to Android Studio"窗口点击"Open an existing Android Studio project",在弹出的路径选择窗口中进入`PaddleX/deploy/lite/android/demo`目录,然后点击右下角的"Open"按钮,导入工程; * 打开Android Studio,在"Welcome to Android Studio"窗口点击"Open an existing Android Studio project",在弹出的路径选择窗口中进入`PaddleX/deploy/lite/android/demo`目录,然后点击右下角的"Open"按钮,导入工程;
...@@ -303,3 +305,58 @@ PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图 ...@@ -303,3 +305,58 @@ PaddleX提供了基于PaddleX Android SDK的安卓demo,可供用户体验图
测试图片及其分割结果如下所示: 测试图片及其分割结果如下所示:
![](./images/beauty.png) ![](./images/beauty.png)
### Nvidia Jetson嵌入式设备部署
#### c++部署
step 1. 下载PaddleX源码
```
git clone https://github.com/PaddlePaddle/PaddleX
```
step 2. 将`PaddleX/examples/human_segmentation/deploy/cpp`下的`human_segmenter.cpp``CMakeList.txt`拷贝至`PaddleX/deploy/cpp`目录下,拷贝之前可以将`PaddleX/deploy/cpp`下原本的`CMakeList.txt`做好备份。
step 3. 按照[Nvidia Jetson开发板部署](../deploy/nvidia-jetson.md)中的Step2至Step3完成C++预测代码的编译。
step 4. 编译成功后,可执行程为`build/human_segmenter`,其主要命令参数说明如下:
| 参数 | 说明 |
| ---- | ---- |
| model_dir | 人像分割模型路径 |
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)|
| gpu_id | GPU 设备ID, 默认值为0 |
| use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) |
| camera_id | 摄像头设备ID,默认值为0 |
| video_path | 视频文件的路径 |
| show_result | 对视频文件做预测时,是否在屏幕上实时显示预测可视化结果,支持值为0或1(默认值为0) |
| save_result | 是否将每帧的预测可视结果保存为视频文件,支持值为0或1(默认值为1) |
| image | 待预测的图片路径 |
| save_dir | 保存可视化结果的路径, 默认值为"output"|
step 5. 推理预测
用于部署推理的模型应为inference格式,本案例使用humanseg_server_inference预训练模型,该模型已经是inference模型,不需要再执行模型导出步骤。如果不使用预训练模型,则执行第2章节`模型训练`中的`模型导出`将自己训练的模型导出为inference格式。
* 使用未加密的模型对单张图片做预测
待测试图片位于本案例提供的测试数据中,可以替换成自己的图片。
```shell
./build/human_segmenter --model_dir=/path/to/humanseg_server_inference --image=/path/to/data/mini_supervisely/Images/pexels-photo-63776.png --use_gpu=1 --save_dir=output
```
* 使用未加密的模型开启摄像头做预测
```shell
./build/human_segmenter --model_dir=/path/to/humanseg_server_inference --use_camera=1 --save_result=1 --use_gpu=1 --save_dir=output
```
* 使用未加密的模型对视频文件做预测
待测试视频文件位于本案例提供的测试数据中,可以替换成自己的视频文件。
```shell
./build/human_segmenter --model_dir=/path/to/humanseg_server_inference --video_path=/path/to/data/mini_supervisely/video_test.mp4 --save_result=1 --use_gpu=1 --save_dir=output
```
...@@ -46,13 +46,13 @@ ...@@ -46,13 +46,13 @@
#### 测试表计读数 #### 测试表计读数
1. 下载PaddleX源码: step 1. 下载PaddleX源码:
``` ```
git clone https://github.com/PaddlePaddle/PaddleX git clone https://github.com/PaddlePaddle/PaddleX
``` ```
2. 预测执行文件位于`PaddleX/examples/meter_reader/`,进入该目录: step 2. 预测执行文件位于`PaddleX/examples/meter_reader/`,进入该目录:
``` ```
cd PaddleX/examples/meter_reader/ cd PaddleX/examples/meter_reader/
...@@ -76,7 +76,7 @@ cd PaddleX/examples/meter_reader/ ...@@ -76,7 +76,7 @@ cd PaddleX/examples/meter_reader/
| use_erode | 是否使用图像腐蚀对分割预测图进行细分,默认为False | | use_erode | 是否使用图像腐蚀对分割预测图进行细分,默认为False |
| erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 | | erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 |
3. 预测 step 3. 预测
若要使用GPU,则指定GPU卡号(以0号卡为例): 若要使用GPU,则指定GPU卡号(以0号卡为例):
...@@ -112,17 +112,17 @@ python3 reader_infer.py --detector_dir /path/to/det_inference_model --segmenter_ ...@@ -112,17 +112,17 @@ python3 reader_infer.py --detector_dir /path/to/det_inference_model --segmenter_
#### c++部署 #### c++部署
1. 下载PaddleX源码: step 1. 下载PaddleX源码:
``` ```
git clone https://github.com/PaddlePaddle/PaddleX git clone https://github.com/PaddlePaddle/PaddleX
``` ```
2.`PaddleX\examples\meter_reader\deploy\cpp`下的`meter_reader`文件夹和`CMakeList.txt`拷贝至`PaddleX\deploy\cpp`目录下,拷贝之前可以将`PaddleX\deploy\cpp`下原本的`CMakeList.txt`做好备份。 step 2. 将`PaddleX\examples\meter_reader\deploy\cpp`下的`meter_reader`文件夹和`CMakeList.txt`拷贝至`PaddleX\deploy\cpp`目录下,拷贝之前可以将`PaddleX\deploy\cpp`下原本的`CMakeList.txt`做好备份。
3. 按照[Windows平台部署](../deploy/server/cpp/windows.md)中的Step2至Step4完成C++预测代码的编译。 step 3. 按照[Windows平台部署](../deploy/server/cpp/windows.md)中的Step2至Step4完成C++预测代码的编译。
4. 编译成功后,可执行文件在`out\build\x64-Release`目录下,打开`cmd`,并切换到该目录: step 4. 编译成功后,可执行文件在`out\build\x64-Release`目录下,打开`cmd`,并切换到该目录:
``` ```
cd PaddleX\deploy\cpp\out\build\x64-Release cd PaddleX\deploy\cpp\out\build\x64-Release
...@@ -139,8 +139,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -139,8 +139,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)| | use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)|
| gpu_id | GPU 设备ID, 默认值为0 | | gpu_id | GPU 设备ID, 默认值为0 |
| save_dir | 保存可视化结果的路径, 默认值为"output"| | save_dir | 保存可视化结果的路径, 默认值为"output"|
| det_key | 检测模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的检测模型 |
| seg_key | 分割模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的分割模型 |
| seg_batch_size | 分割的批量大小,默认为2 | | seg_batch_size | 分割的批量大小,默认为2 |
| thread_num | 分割预测的线程数,默认为cpu处理器个数 | | thread_num | 分割预测的线程数,默认为cpu处理器个数 |
| use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) | | use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) |
...@@ -149,7 +147,7 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -149,7 +147,7 @@ git clone https://github.com/PaddlePaddle/PaddleX
| erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 | | erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 |
| score_threshold | 检测模型输出结果中,预测得分低于该阈值的框将被滤除,默认值为0.5| | score_threshold | 检测模型输出结果中,预测得分低于该阈值的框将被滤除,默认值为0.5|
5. 推理预测: step 5. 推理预测:
用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[部署模型导出](../deploy/export_model.md)将模型导出为inference格式。 用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[部署模型导出](../deploy/export_model.md)将模型导出为inference格式。
...@@ -160,6 +158,13 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -160,6 +158,13 @@ git clone https://github.com/PaddlePaddle/PaddleX
``` ```
* 使用未加密的模型对图像列表做预测 * 使用未加密的模型对图像列表做预测
图像列表image_list.txt内容的格式如下,因绝对路径不同,暂未提供该文件,用户可根据实际情况自行生成:
```
\path\to\images\1.jpg
\path\to\images\2.jpg
...
\path\to\images\n.jpg
```
```shell ```shell
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image_list=\path\to\meter_test\image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image_list=\path\to\meter_test\image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output
...@@ -171,29 +176,29 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -171,29 +176,29 @@ git clone https://github.com/PaddlePaddle/PaddleX
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
* 使用加密后的模型对单张图片做预测 * 使用加密后的模型对单张图片做预测
如果未对模型进行加密,请参考[加密PaddleX模型](../deploy/server/encryption.html#paddlex)对模型进行加密。例如加密后的检测模型所在目录为`\path\to\encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`\path\to\encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=` 如果未对模型进行加密,请参考[加密PaddleX模型](../deploy/server/encryption.html#paddlex)对模型进行加密。例如加密后的检测模型所在目录为`\path\to\encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`\path\to\encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=`
```shell ```shell
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\encrypted_det_inference_model --seg_model_dir=\path\to\encrypted_seg_inference_model --image=\path\to\test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY= .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\encrypted_det_inference_model --seg_model_dir=\path\to\encrypted_seg_inference_model --image=\path\to\test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=
``` ```
### Linux系统的jetson嵌入式设备安全部署 ### Linux系统的jetson嵌入式设备安全部署
#### c++部署 #### c++部署
1. 下载PaddleX源码: step 1. 下载PaddleX源码:
``` ```
git clone https://github.com/PaddlePaddle/PaddleX git clone https://github.com/PaddlePaddle/PaddleX
``` ```
2.`PaddleX/examples/meter_reader/deploy/cpp`下的`meter_reader`文件夹和`CMakeList.txt`拷贝至`PaddleX/deploy/cpp`目录下,拷贝之前可以将`PaddleX/deploy/cpp`下原本的`CMakeList.txt`做好备份。 step 2. 将`PaddleX/examples/meter_reader/deploy/cpp`下的`meter_reader`文件夹和`CMakeList.txt`拷贝至`PaddleX/deploy/cpp`目录下,拷贝之前可以将`PaddleX/deploy/cpp`下原本的`CMakeList.txt`做好备份。
3. 按照[Nvidia Jetson开发板部署](../deploy/nvidia-jetson.md)中的Step2至Step3完成C++预测代码的编译。 step 3. 按照[Nvidia Jetson开发板部署](../deploy/nvidia-jetson.md)中的Step2至Step3完成C++预测代码的编译。
4. 编译成功后,可执行程为`build/meter_reader/meter_reader`,其主要命令参数说明如下: step 4. 编译成功后,可执行程为`build/meter_reader/meter_reader`,其主要命令参数说明如下:
| 参数 | 说明 | | 参数 | 说明 |
| ---- | ---- | | ---- | ---- |
...@@ -204,8 +209,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -204,8 +209,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)| | use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)|
| gpu_id | GPU 设备ID, 默认值为0 | | gpu_id | GPU 设备ID, 默认值为0 |
| save_dir | 保存可视化结果的路径, 默认值为"output"| | save_dir | 保存可视化结果的路径, 默认值为"output"|
| det_key | 检测模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的检测模型 |
| seg_key | 分割模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的分割模型 |
| seg_batch_size | 分割的批量大小,默认为2 | | seg_batch_size | 分割的批量大小,默认为2 |
| thread_num | 分割预测的线程数,默认为cpu处理器个数 | | thread_num | 分割预测的线程数,默认为cpu处理器个数 |
| use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) | | use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) |
...@@ -214,7 +217,7 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -214,7 +217,7 @@ git clone https://github.com/PaddlePaddle/PaddleX
| erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 | | erode_kernel | 图像腐蚀操作时的卷积核大小,默认值为4 |
| score_threshold | 检测模型输出结果中,预测得分低于该阈值的框将被滤除,默认值为0.5| | score_threshold | 检测模型输出结果中,预测得分低于该阈值的框将被滤除,默认值为0.5|
5. 推理预测: step 5. 推理预测:
用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[部署模型导出](../deploy/export_model.md)将模型导出为inference格式。 用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[部署模型导出](../deploy/export_model.md)将模型导出为inference格式。
...@@ -225,7 +228,13 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -225,7 +228,13 @@ git clone https://github.com/PaddlePaddle/PaddleX
``` ```
* 使用未加密的模型对图像列表做预测 * 使用未加密的模型对图像列表做预测
图像列表image_list.txt内容的格式如下,因绝对路径不同,暂未提供该文件,用户可根据实际情况自行生成:
```
\path\to\images\1.jpg
\path\to\images\2.jpg
...
\path\to\images\n.jpg
```
```shell ```shell
./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --image_list=/path/to/image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output ./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --image_list=/path/to/image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
...@@ -236,15 +245,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -236,15 +245,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output ./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
* 使用加密后的模型对单张图片做预测
如果未对模型进行加密,请参考[加密PaddleX模型](../deploy/server/encryption.html#paddlex)对模型进行加密。例如加密后的检测模型所在目录为`/path/to/encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`/path/to/encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=`
```shell
./build/meter_reader/meter_reader --det_model_dir=/path/to/encrypted_det_inference_model --seg_model_dir=/path/to/encrypted_seg_inference_model --image=/path/to/test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=
```
## 模型训练 ## 模型训练
......
...@@ -42,6 +42,7 @@ PaddleX针对图像分类、目标检测、实例分割和语义分割4种视觉 ...@@ -42,6 +42,7 @@ PaddleX针对图像分类、目标检测、实例分割和语义分割4种视觉
| YOLOv3-MobileNetV3_larget | 适用于追求高速预测的移动端场景 | 100.7MB | 143.322 | - | - | 31.6 | | YOLOv3-MobileNetV3_larget | 适用于追求高速预测的移动端场景 | 100.7MB | 143.322 | - | - | 31.6 |
| YOLOv3-MobileNetV1 | 精度相对偏低,适用于追求高速预测的服务器端场景 | 99.2MB| 15.422 | - | - | 29.3 | | YOLOv3-MobileNetV1 | 精度相对偏低,适用于追求高速预测的服务器端场景 | 99.2MB| 15.422 | - | - | 29.3 |
| YOLOv3-DarkNet53 | 在预测速度和模型精度上都有较好的表现,适用于大多数的服务器端场景| 249.2MB | 42.672 | - | - | 38.9 | | YOLOv3-DarkNet53 | 在预测速度和模型精度上都有较好的表现,适用于大多数的服务器端场景| 249.2MB | 42.672 | - | - | 38.9 |
| PPYOLO | 预测速度和模型精度都比YOLOv3-DarkNet53优异,适用于大多数的服务器端场景 | 329.1MB | - | - | - | 45.9 |
| FasterRCNN-ResNet50-FPN | 经典的二阶段检测器,预测速度相对较慢,适用于重视模型精度的服务器端场景 | 167.MB | 83.189 | - | -| 37.2 | | FasterRCNN-ResNet50-FPN | 经典的二阶段检测器,预测速度相对较慢,适用于重视模型精度的服务器端场景 | 167.MB | 83.189 | - | -| 37.2 |
| FasterRCNN-HRNet_W18-FPN | 适用于对图像分辨率较为敏感、对目标细节预测要求更高的服务器端场景 | 115.5MB | 81.592 | - | - | 36 | | FasterRCNN-HRNet_W18-FPN | 适用于对图像分辨率较为敏感、对目标细节预测要求更高的服务器端场景 | 115.5MB | 81.592 | - | - | 36 |
| FasterRCNN-ResNet101_vd-FPN | 超高精度模型,预测时间更长,在处理较大数据量时有较高的精度,适用于服务器端场景 | 244.3MB | 156.097 | - | - | 40.5 | | FasterRCNN-ResNet101_vd-FPN | 超高精度模型,预测时间更长,在处理较大数据量时有较高的精度,适用于服务器端场景 | 244.3MB | 156.097 | - | - | 40.5 |
...@@ -74,11 +75,12 @@ PaddleX目前提供了实例分割MaskRCNN模型,支持5种不同的backbone ...@@ -74,11 +75,12 @@ PaddleX目前提供了实例分割MaskRCNN模型,支持5种不同的backbone
> 表中GPU预测速度是使用PaddlePaddle Python预测接口测试得到(测试GPU型号为Nvidia Tesla P40)。 > 表中GPU预测速度是使用PaddlePaddle Python预测接口测试得到(测试GPU型号为Nvidia Tesla P40)。
> 表中CPU预测速度 (测试CPU型号为)。 > 表中CPU预测速度 (测试CPU型号为)。
> 表中骁龙855预测速度是使用处理器为骁龙855的手机测试得到。 > 表中骁龙855预测速度是使用处理器为骁龙855的手机测试得到。
> 测速时模型的输入大小为1024 x 2048,mIOU为Cityscapes数据集上评估所得。 > 测速时模型的输入大小为1024 x 2048,mIoU为Cityscapes数据集上评估所得。
| 模型 | 模型特点 | 存储体积 | GPU预测速度 | CPU(x86)预测速度(毫秒) | 骁龙855(ARM)预测速度 (毫秒)| mIOU | | 模型 | 模型特点 | 存储体积 | GPU预测速度 | CPU(x86)预测速度(毫秒) | 骁龙855(ARM)预测速度 (毫秒)| mIoU |
| :---- | :------- | :---------- | :---------- | :----- | :----- |:--- | | :---- | :------- | :---------- | :---------- | :----- | :----- |:--- |
| DeepLabv3p-MobileNetV2_x1.0 | 轻量级模型,适用于移动端场景| - | - | - | 69.8% | | DeepLabv3p-MobileNetV2_x1.0 | 轻量级模型,适用于移动端场景| - | - | - | 69.8% |
| DeepLabv3-MobileNetV3_large_x1_0_ssld | 轻量级模型,适用于移动端场景| - | - | - | 73.28% |
| HRNet_W18_Small_v1 | 轻量高速,适用于移动端场景 | - | - | - | - | | HRNet_W18_Small_v1 | 轻量高速,适用于移动端场景 | - | - | - | - |
| FastSCNN | 轻量高速,适用于追求高速预测的移动端或服务器端场景 | - | - | - | 69.64 | | FastSCNN | 轻量高速,适用于追求高速预测的移动端或服务器端场景 | - | - | - | 69.64 |
| HRNet_W18 | 高精度模型,适用于对图像分辨率较为敏感、对目标细节预测要求更高的服务器端场景| - | - | - | 79.36 | | HRNet_W18 | 高精度模型,适用于对图像分辨率较为敏感、对目标细节预测要求更高的服务器端场景| - | - | - | 79.36 |
......
...@@ -33,4 +33,4 @@ ...@@ -33,4 +33,4 @@
**如果您有任何问题或建议,欢迎以issue的形式,或加入PaddleX官方QQ群(1045148026)直接反馈您的问题和需求** **如果您有任何问题或建议,欢迎以issue的形式,或加入PaddleX官方QQ群(1045148026)直接反馈您的问题和需求**
![](/Users/lvxueying/Documents/LaraPaddleX/docs/paddlex_gui/images/QR.jpg) ![](./images/QR.jpg)
\ No newline at end of file
...@@ -29,4 +29,4 @@ python mobilenetv3_small_ssld.py ...@@ -29,4 +29,4 @@ python mobilenetv3_small_ssld.py
-**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md) -**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md)
-**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925) -**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925)
-**拓展**】更多图像分类模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/index.html) -**拓展**】更多图像分类模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/classification.md)
...@@ -13,3 +13,4 @@ PaddleX集成了PaddleClas、PaddleDetection和PaddleSeg三大CV工具套件中 ...@@ -13,3 +13,4 @@ PaddleX集成了PaddleClas、PaddleDetection和PaddleSeg三大CV工具套件中
instance_segmentation.md instance_segmentation.md
semantic_segmentation.md semantic_segmentation.md
prediction.md prediction.md
visualdl.md
...@@ -27,4 +27,4 @@ python mask_rcnn_r50_fpn.py ...@@ -27,4 +27,4 @@ python mask_rcnn_r50_fpn.py
-**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md) -**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md)
-**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925) -**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925)
-**拓展**】更多实例分割模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/index.html) -**拓展**】更多实例分割模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/instance_segmentation.md)
...@@ -13,6 +13,7 @@ PaddleX目前提供了FasterRCNN和YOLOv3两种检测结构,多种backbone模 ...@@ -13,6 +13,7 @@ PaddleX目前提供了FasterRCNN和YOLOv3两种检测结构,多种backbone模
| [YOLOv3-MobileNetV1](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_mobilenetv1.py) | 29.3% | 99.2MB | 15.442ms | - | 模型小,预测速度快,适用于低性能或移动端设备 | | [YOLOv3-MobileNetV1](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_mobilenetv1.py) | 29.3% | 99.2MB | 15.442ms | - | 模型小,预测速度快,适用于低性能或移动端设备 |
| [YOLOv3-MobileNetV3](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_mobilenetv3.py) | 31.6% | 100.7MB | 143.322ms | - | 模型小,移动端上预测速度有优势 | | [YOLOv3-MobileNetV3](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_mobilenetv3.py) | 31.6% | 100.7MB | 143.322ms | - | 模型小,移动端上预测速度有优势 |
| [YOLOv3-DarkNet53](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_darknet53.py) | 38.9% | 249.2MB | 42.672ms | - | 模型较大,预测速度快,适用于服务端 | | [YOLOv3-DarkNet53](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/yolov3_darknet53.py) | 38.9% | 249.2MB | 42.672ms | - | 模型较大,预测速度快,适用于服务端 |
| [PPYOLO](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/ppyolo.py) | 45.9% | 329.1MB | - | - | 模型较大,预测速度比YOLOv3-DarkNet53更快,适用于服务端 |
| [FasterRCNN-ResNet50-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_r50_fpn.py) | 37.2% | 167.7MB | 197.715ms | - | 模型精度高,适用于服务端部署 | | [FasterRCNN-ResNet50-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_r50_fpn.py) | 37.2% | 167.7MB | 197.715ms | - | 模型精度高,适用于服务端部署 |
| [FasterRCNN-ResNet18-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_r18_fpn.py) | 32.6% | 173.2MB | - | - | 模型精度高,适用于服务端部署 | | [FasterRCNN-ResNet18-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_r18_fpn.py) | 32.6% | 173.2MB | - | - | 模型精度高,适用于服务端部署 |
| [FasterRCNN-HRNet-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_hrnet_fpn.py) | 36.0% | 115.MB | 81.592ms | - | 模型精度高,预测速度快,适用于服务端部署 | | [FasterRCNN-HRNet-FPN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/object_detection/faster_rcnn_hrnet_fpn.py) | 36.0% | 115.MB | 81.592ms | - | 模型精度高,预测速度快,适用于服务端部署 |
...@@ -31,4 +32,4 @@ python yolov3_mobilenetv1.py ...@@ -31,4 +32,4 @@ python yolov3_mobilenetv1.py
-**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md) -**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md)
-**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925) -**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925)
-**拓展**】更多目标检测模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/index.html) -**拓展**】更多目标检测模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/detection.md)
...@@ -4,15 +4,16 @@ ...@@ -4,15 +4,16 @@
PaddleX目前提供了DeepLabv3p、UNet、HRNet和FastSCNN四种语义分割结构,多种backbone模型,可满足开发者不同场景和性能的需求。 PaddleX目前提供了DeepLabv3p、UNet、HRNet和FastSCNN四种语义分割结构,多种backbone模型,可满足开发者不同场景和性能的需求。
- **mIOU**: 模型在CityScape数据集上的测试精度 - **mIoU**: 模型在CityScape数据集上的测试精度
- **预测速度**:单张图片的预测用时(不包括预处理和后处理) - **预测速度**:单张图片的预测用时(不包括预处理和后处理)
- "-"表示指标暂未更新 - "-"表示指标暂未更新
| 模型(点击获取代码) | mIOU | 模型大小 | GPU预测速度 | Arm预测速度 | 备注 | | 模型(点击获取代码) | mIoU | 模型大小 | GPU预测速度 | Arm预测速度 | 备注 |
| :---------------- | :------- | :------- | :--------- | :--------- | :----- | | :---------------- | :------- | :------- | :--------- | :--------- | :----- |
| [DeepLabv3p-MobileNetV2-x0.25](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_mobilenetv2_x0.25.py) | - | 2.9MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 | | [DeepLabv3p-MobileNetV2-x0.25](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_mobilenetv2_x0.25.py) | - | 2.9MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 |
| [DeepLabv3p-MobileNetV2-x1.0](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_mobilenetv2.py) | 69.8% | 11MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 | | [DeepLabv3p-MobileNetV2-x1.0](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_mobilenetv2.py) | 69.8% | 11MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 |
| [DeepLabv3p-Xception65](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_xception65.pyy) | 79.3% | 158MB | - | - | 模型大,精度高,适用于服务端 | | [DeepLabv3_MobileNetV3_large_x1_0_ssld](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_mobilenetv3_large_ssld.py) | 73.28% | 9.3MB | - | - | 模型小,预测速度快,精度较高,适用于低性能或移动端设备 |
| [DeepLabv3p-Xception65](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/deeplabv3p_xception65.py) | 79.3% | 158MB | - | - | 模型大,精度高,适用于服务端 |
| [UNet](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/unet.py) | - | 52MB | - | - | 模型较大,精度高,适用于服务端 | | [UNet](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/unet.py) | - | 52MB | - | - | 模型较大,精度高,适用于服务端 |
| [HRNet](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/hrnet.py) | 79.4% | 37MB | - | - | 模型较小,模型精度高,适用于服务端部署 | | [HRNet](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/hrnet.py) | 79.4% | 37MB | - | - | 模型较小,模型精度高,适用于服务端部署 |
| [FastSCNN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/fast_scnn.py) | - | 4.5MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 | | [FastSCNN](https://github.com/PaddlePaddle/PaddleX/blob/develop/tutorials/train/semantic_segmentation/fast_scnn.py) | - | 4.5MB | - | - | 模型小,预测速度快,适用于低性能或移动端设备 |
...@@ -30,4 +31,4 @@ python deeplabv3p_mobilenetv2_x0.25.py ...@@ -30,4 +31,4 @@ python deeplabv3p_mobilenetv2_x0.25.py
-**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md) -**重要**】针对自己的机器环境和数据,调整训练参数?先了解下PaddleX中训练参数作用。[——>>传送门](../appendix/parameters.md)
-**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925) -**有用**】没有机器资源?使用AIStudio免费的GPU资源在线训练模型。[——>>传送门](https://aistudio.baidu.com/aistudio/projectdetail/450925)
-**拓展**】更多语义分割模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/index.html) -**拓展**】更多语义分割模型,查阅[PaddleX模型库](../appendix/model_zoo.md)[API使用文档](../apis/models/semantic_segmentation.md)
# VisualDL可视化训练指标
在使用PaddleX训练模型过程中,各个训练指标和评估指标会直接输出到标准输出流,同时也可通过VisualDL对训练过程中的指标进行可视化,只需在调用`train`函数时,将`use_vdl`参数设为`True`即可,如下代码所示,
```
model = paddlex.cls.ResNet50(num_classes=1000)
model.train(num_epochs=120, train_dataset=train_dataset,
train_batch_size=32, eval_dataset=eval_dataset,
log_interval_steps=10, save_interval_epochs=10,
save_dir='./output', use_vdl=True)
```
模型在训练过程中,会在`save_dir`下生成`vdl_log`目录,通过在命令行终端执行以下命令,启动VisualDL。
```
visualdl --logdir=output/vdl_log --port=8008
```
在浏览器打开`http://0.0.0.0:8008`便可直接查看随训练迭代动态变化的各个指标(0.0.0.0表示启动VisualDL所在服务器的IP,本机使用0.0.0.0即可)。
在训练分类模型过程中,使用VisualDL进行可视化的示例图如下所示。
> 训练过程中每个Step的`Loss`和相应`Top1准确率`变化趋势:
![](../images/vdl1.jpg)
> 训练过程中每个Step的`学习率lr`和相应`Top5准确率`变化趋势:
![](../images/vdl2.jpg)
> 训练过程中,每次保存模型时,模型在验证数据集上的`Top1准确率`和`Top5准确率`:
![](../images/vdl3.jpg)
cmake_minimum_required(VERSION 3.0)
project(PaddleX CXX C)
option(WITH_MKL "Compile human_segmenter with MKL/OpenBlas support,defaultuseMKL." ON)
option(WITH_GPU "Compile human_segmenter with GPU/CPU, default use CPU." ON)
if (NOT WIN32)
option(WITH_STATIC_LIB "Compile human_segmenter with static/shared library, default use static." OFF)
else()
option(WITH_STATIC_LIB "Compile human_segmenter with static/shared library, default use static." ON)
endif()
option(WITH_TENSORRT "Compile human_segmenter with TensorRT." OFF)
option(WITH_ENCRYPTION "Compile human_segmenter with encryption tool." OFF)
SET(TENSORRT_DIR "" CACHE PATH "Location of libraries")
SET(PADDLE_DIR "" CACHE PATH "Location of libraries")
SET(OPENCV_DIR "" CACHE PATH "Location of libraries")
SET(ENCRYPTION_DIR"" CACHE PATH "Location of libraries")
SET(CUDA_LIB "" CACHE PATH "Location of libraries")
if (NOT WIN32)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
else()
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/paddlex_inference)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/paddlex_inference)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/paddlex_inference)
endif()
if (NOT WIN32)
SET(YAML_BUILD_TYPE ON CACHE BOOL "yaml build shared library.")
else()
SET(YAML_BUILD_TYPE OFF CACHE BOOL "yaml build shared library.")
endif()
include(cmake/yaml-cpp.cmake)
include_directories("${CMAKE_SOURCE_DIR}/")
include_directories("${CMAKE_CURRENT_BINARY_DIR}/ext/yaml-cpp/src/ext-yaml-cpp/include")
link_directories("${CMAKE_CURRENT_BINARY_DIR}/ext/yaml-cpp/lib")
macro(safe_set_static_flag)
foreach(flag_var
CMAKE_CXX_FLAGS CMAKE_CXX_FLAGS_DEBUG CMAKE_CXX_FLAGS_RELEASE
CMAKE_CXX_FLAGS_MINSIZEREL CMAKE_CXX_FLAGS_RELWITHDEBINFO)
if(${flag_var} MATCHES "/MD")
string(REGEX REPLACE "/MD" "/MT" ${flag_var} "${${flag_var}}")
endif(${flag_var} MATCHES "/MD")
endforeach(flag_var)
endmacro()
if (WITH_ENCRYPTION)
add_definitions( -DWITH_ENCRYPTION=${WITH_ENCRYPTION})
endif()
if (WITH_MKL)
ADD_DEFINITIONS(-DUSE_MKL)
endif()
if (NOT DEFINED PADDLE_DIR OR ${PADDLE_DIR} STREQUAL "")
message(FATAL_ERROR "please set PADDLE_DIR with -DPADDLE_DIR=/path/paddle_influence_dir")
endif()
if (NOT (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64"))
if (NOT DEFINED OPENCV_DIR OR ${OPENCV_DIR} STREQUAL "")
message(FATAL_ERROR "please set OPENCV_DIR with -DOPENCV_DIR=/path/opencv")
endif()
endif()
include_directories("${CMAKE_SOURCE_DIR}/")
include_directories("${PADDLE_DIR}/")
include_directories("${PADDLE_DIR}/third_party/install/protobuf/include")
include_directories("${PADDLE_DIR}/third_party/install/glog/include")
include_directories("${PADDLE_DIR}/third_party/install/gflags/include")
include_directories("${PADDLE_DIR}/third_party/install/xxhash/include")
if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/include")
include_directories("${PADDLE_DIR}/third_party/install/snappy/include")
endif()
if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/include")
include_directories("${PADDLE_DIR}/third_party/install/snappystream/include")
endif()
# zlib does not exist in 1.8.1
if (EXISTS "${PADDLE_DIR}/third_party/install/zlib/include")
include_directories("${PADDLE_DIR}/third_party/install/zlib/include")
endif()
include_directories("${PADDLE_DIR}/third_party/boost")
include_directories("${PADDLE_DIR}/third_party/eigen3")
if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib")
link_directories("${PADDLE_DIR}/third_party/install/snappy/lib")
endif()
if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib")
link_directories("${PADDLE_DIR}/third_party/install/snappystream/lib")
endif()
if (EXISTS "${PADDLE_DIR}/third_party/install/zlib/lib")
link_directories("${PADDLE_DIR}/third_party/install/zlib/lib")
endif()
link_directories("${PADDLE_DIR}/third_party/install/protobuf/lib")
link_directories("${PADDLE_DIR}/third_party/install/glog/lib")
link_directories("${PADDLE_DIR}/third_party/install/gflags/lib")
link_directories("${PADDLE_DIR}/third_party/install/xxhash/lib")
link_directories("${PADDLE_DIR}/paddle/lib/")
link_directories("${CMAKE_CURRENT_BINARY_DIR}")
if (WIN32)
include_directories("${PADDLE_DIR}/paddle/fluid/inference")
include_directories("${PADDLE_DIR}/paddle/include")
link_directories("${PADDLE_DIR}/paddle/fluid/inference")
find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/build/ NO_DEFAULT_PATH)
unset(OpenCV_DIR CACHE)
else ()
if (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64") # x86_64 aarch64
set(OpenCV_INCLUDE_DIRS "/usr/include/opencv4")
file(GLOB OpenCV_LIBS /usr/lib/aarch64-linux-gnu/libopencv_*${CMAKE_SHARED_LIBRARY_SUFFIX})
message("OpenCV libs: ${OpenCV_LIBS}")
else()
find_package(OpenCV REQUIRED PATHS ${OPENCV_DIR}/share/OpenCV NO_DEFAULT_PATH)
endif()
include_directories("${PADDLE_DIR}/paddle/include")
link_directories("${PADDLE_DIR}/paddle/lib")
endif ()
include_directories(${OpenCV_INCLUDE_DIRS})
if (WIN32)
add_definitions("/DGOOGLE_GLOG_DLL_DECL=")
find_package(OpenMP REQUIRED)
if (OPENMP_FOUND)
message("OPENMP FOUND")
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} ${OpenMP_C_FLAGS}")
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} ${OpenMP_C_FLAGS}")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} ${OpenMP_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${OpenMP_CXX_FLAGS}")
endif()
set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /bigobj /MTd")
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} /bigobj /MT")
set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /bigobj /MTd")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /bigobj /MT")
if (WITH_STATIC_LIB)
safe_set_static_flag()
add_definitions(-DSTATIC_LIB)
endif()
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -o2 -fopenmp -std=c++11")
set(CMAKE_STATIC_LIBRARY_PREFIX "")
endif()
if (WITH_GPU)
if (NOT DEFINED CUDA_LIB OR ${CUDA_LIB} STREQUAL "")
message(FATAL_ERROR "please set CUDA_LIB with -DCUDA_LIB=/path/cuda/lib64")
endif()
if (NOT WIN32)
if (NOT DEFINED CUDNN_LIB)
message(FATAL_ERROR "please set CUDNN_LIB with -DCUDNN_LIB=/path/cudnn/")
endif()
endif(NOT WIN32)
endif()
if (NOT WIN32)
if (WITH_TENSORRT AND WITH_GPU)
include_directories("${TENSORRT_DIR}/include")
link_directories("${TENSORRT_DIR}/lib")
endif()
endif(NOT WIN32)
if (NOT WIN32)
set(NGRAPH_PATH "${PADDLE_DIR}/third_party/install/ngraph")
if(EXISTS ${NGRAPH_PATH})
include(GNUInstallDirs)
include_directories("${NGRAPH_PATH}/include")
link_directories("${NGRAPH_PATH}/${CMAKE_INSTALL_LIBDIR}")
set(NGRAPH_LIB ${NGRAPH_PATH}/${CMAKE_INSTALL_LIBDIR}/libngraph${CMAKE_SHARED_LIBRARY_SUFFIX})
endif()
endif()
if(WITH_MKL)
include_directories("${PADDLE_DIR}/third_party/install/mklml/include")
if (WIN32)
set(MATH_LIB ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.lib
${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.lib)
else ()
set(MATH_LIB ${PADDLE_DIR}/third_party/install/mklml/lib/libmklml_intel${CMAKE_SHARED_LIBRARY_SUFFIX}
${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5${CMAKE_SHARED_LIBRARY_SUFFIX})
execute_process(COMMAND cp -r ${PADDLE_DIR}/third_party/install/mklml/lib/libmklml_intel${CMAKE_SHARED_LIBRARY_SUFFIX} /usr/lib)
endif ()
set(MKLDNN_PATH "${PADDLE_DIR}/third_party/install/mkldnn")
if(EXISTS ${MKLDNN_PATH})
include_directories("${MKLDNN_PATH}/include")
if (WIN32)
set(MKLDNN_LIB ${MKLDNN_PATH}/lib/mkldnn.lib)
else ()
set(MKLDNN_LIB ${MKLDNN_PATH}/lib/libmkldnn.so.0)
endif ()
endif()
else()
set(MATH_LIB ${PADDLE_DIR}/third_party/install/openblas/lib/libopenblas${CMAKE_STATIC_LIBRARY_SUFFIX})
endif()
if (WIN32)
if(EXISTS "${PADDLE_DIR}/paddle/fluid/inference/libpaddle_fluid${CMAKE_STATIC_LIBRARY_SUFFIX}")
set(DEPS
${PADDLE_DIR}/paddle/fluid/inference/libpaddle_fluid${CMAKE_STATIC_LIBRARY_SUFFIX})
else()
set(DEPS
${PADDLE_DIR}/paddle/lib/libpaddle_fluid${CMAKE_STATIC_LIBRARY_SUFFIX})
endif()
endif()
if(WITH_STATIC_LIB)
set(DEPS
${PADDLE_DIR}/paddle/lib/libpaddle_fluid${CMAKE_STATIC_LIBRARY_SUFFIX})
else()
if (NOT WIN32)
set(DEPS
${PADDLE_DIR}/paddle/lib/libpaddle_fluid${CMAKE_SHARED_LIBRARY_SUFFIX})
else()
set(DEPS
${PADDLE_DIR}/paddle/lib/paddle_fluid${CMAKE_SHARED_LIBRARY_SUFFIX})
endif()
endif()
if (NOT WIN32)
set(DEPS ${DEPS}
${MATH_LIB} ${MKLDNN_LIB}
glog gflags protobuf z xxhash yaml-cpp
)
if(EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib")
set(DEPS ${DEPS} snappystream)
endif()
if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib")
set(DEPS ${DEPS} snappy)
endif()
else()
set(DEPS ${DEPS}
${MATH_LIB} ${MKLDNN_LIB}
glog gflags_static libprotobuf xxhash libyaml-cppmt)
if (EXISTS "${PADDLE_DIR}/third_party/install/zlib/lib")
set(DEPS ${DEPS} zlibstatic)
endif()
set(DEPS ${DEPS} libcmt shlwapi)
if (EXISTS "${PADDLE_DIR}/third_party/install/snappy/lib")
set(DEPS ${DEPS} snappy)
endif()
if (EXISTS "${PADDLE_DIR}/third_party/install/snappystream/lib")
set(DEPS ${DEPS} snappystream)
endif()
endif(NOT WIN32)
if(WITH_GPU)
if(NOT WIN32)
if (WITH_TENSORRT)
set(DEPS ${DEPS} ${TENSORRT_DIR}/lib/libnvinfer${CMAKE_SHARED_LIBRARY_SUFFIX})
set(DEPS ${DEPS} ${TENSORRT_DIR}/lib/libnvinfer_plugin${CMAKE_SHARED_LIBRARY_SUFFIX})
endif()
set(DEPS ${DEPS} ${CUDA_LIB}/libcudart${CMAKE_SHARED_LIBRARY_SUFFIX})
set(DEPS ${DEPS} ${CUDNN_LIB}/libcudnn${CMAKE_SHARED_LIBRARY_SUFFIX})
else()
set(DEPS ${DEPS} ${CUDA_LIB}/cudart${CMAKE_STATIC_LIBRARY_SUFFIX} )
set(DEPS ${DEPS} ${CUDA_LIB}/cublas${CMAKE_STATIC_LIBRARY_SUFFIX} )
set(DEPS ${DEPS} ${CUDA_LIB}/cudnn${CMAKE_STATIC_LIBRARY_SUFFIX})
endif()
endif()
if(WITH_ENCRYPTION)
if(NOT WIN32)
include_directories("${ENCRYPTION_DIR}/include")
link_directories("${ENCRYPTION_DIR}/lib")
set(DEPS ${DEPS} ${ENCRYPTION_DIR}/lib/libpmodel-decrypt${CMAKE_SHARED_LIBRARY_SUFFIX})
else()
include_directories("${ENCRYPTION_DIR}/include")
link_directories("${ENCRYPTION_DIR}/lib")
set(DEPS ${DEPS} ${ENCRYPTION_DIR}/lib/pmodel-decrypt${CMAKE_STATIC_LIBRARY_SUFFIX})
endif()
endif()
if (NOT WIN32)
set(EXTERNAL_LIB "-ldl -lrt -lgomp -lz -lm -lpthread")
set(DEPS ${DEPS} ${EXTERNAL_LIB})
endif()
set(DEPS ${DEPS} ${OpenCV_LIBS})
add_library(paddlex_inference SHARED src/visualize src/transforms.cpp src/paddlex.cpp)
ADD_DEPENDENCIES(paddlex_inference ext-yaml-cpp)
target_link_libraries(paddlex_inference ${DEPS})
add_executable(human_segmenter human_segmenter.cpp src/transforms.cpp src/paddlex.cpp src/visualize.cpp)
ADD_DEPENDENCIES(human_segmenter ext-yaml-cpp)
target_link_libraries(human_segmenter ${DEPS})
if (WIN32 AND WITH_MKL)
add_custom_command(TARGET human_segmenter POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./mkldnn.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/mklml.dll ./release/mklml.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mklml/lib/libiomp5md.dll ./release/libiomp5md.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${PADDLE_DIR}/third_party/install/mkldnn/lib/mkldnn.dll ./release/mkldnn.dll
)
# for encryption
if (EXISTS "${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll")
add_custom_command(TARGET human_segmenter POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./pmodel-decrypt.dll
COMMAND ${CMAKE_COMMAND} -E copy_if_different ${ENCRYPTION_DIR}/lib/pmodel-decrypt.dll ./release/pmodel-decrypt.dll
)
endif()
endif()
file(COPY "${CMAKE_SOURCE_DIR}/include/paddlex/visualize.h"
DESTINATION "${CMAKE_BINARY_DIR}/include/" )
file(COPY "${CMAKE_SOURCE_DIR}/include/paddlex/config_parser.h"
DESTINATION "${CMAKE_BINARY_DIR}/include/" )
file(COPY "${CMAKE_SOURCE_DIR}/include/paddlex/transforms.h"
DESTINATION "${CMAKE_BINARY_DIR}/include/" )
file(COPY "${CMAKE_SOURCE_DIR}/include/paddlex/results.h"
DESTINATION "${CMAKE_BINARY_DIR}/include/" )
file(COPY "${CMAKE_SOURCE_DIR}/include/paddlex/paddlex.h"
DESTINATION "${CMAKE_BINARY_DIR}/include/" )
// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include <glog/logging.h>
#include <omp.h>
#include <algorithm>
#include <chrono> // NOLINT
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <utility>
#include <ctime>
#include "include/paddlex/paddlex.h"
#include "include/paddlex/visualize.h"
#if defined(__arm__) || defined(__aarch64__)
#include <opencv2/videoio/legacy/constants_c.h>
#endif
using namespace std::chrono; // NOLINT
DEFINE_string(model_dir, "", "Path of inference model");
DEFINE_bool(use_gpu, false, "Infering with GPU or CPU");
DEFINE_bool(use_trt, false, "Infering with TensorRT");
DEFINE_int32(gpu_id, 0, "GPU card id");
DEFINE_string(key, "", "key of encryption");
DEFINE_string(image, "", "Path of test image file");
DEFINE_bool(use_camera, false, "Infering with Camera");
DEFINE_int32(camera_id, 0, "Camera id");
DEFINE_string(video_path, "", "Path of input video");
DEFINE_bool(show_result, false, "show the result of each frame with a window");
DEFINE_bool(save_result, true, "save the result of each frame to a video");
DEFINE_string(save_dir, "output", "Path to save visualized image");
int main(int argc, char** argv) {
// Parsing command-line
google::ParseCommandLineFlags(&argc, &argv, true);
if (FLAGS_model_dir == "") {
std::cerr << "--model_dir need to be defined" << std::endl;
return -1;
}
if (FLAGS_image == "" & FLAGS_video_path == ""
& FLAGS_use_camera == false) {
std::cerr << "--image or --video_path or --use_camera need to be defined"
<< std::endl;
return -1;
}
// Load model
PaddleX::Model model;
model.Init(FLAGS_model_dir,
FLAGS_use_gpu,
FLAGS_use_trt,
FLAGS_gpu_id,
FLAGS_key);
if (FLAGS_use_camera || FLAGS_video_path != "") {
// Open video
cv::VideoCapture capture;
if (FLAGS_use_camera) {
capture.open(FLAGS_camera_id);
if (!capture.isOpened()) {
std::cout << "Can not open the camera "
<< FLAGS_camera_id << "."
<< std::endl;
return -1;
}
} else {
capture.open(FLAGS_video_path);
if (!capture.isOpened()) {
std::cout << "Can not open the video "
<< FLAGS_video_path << "."
<< std::endl;
return -1;
}
}
// Create a VideoWriter
cv::VideoWriter video_out;
std::string video_out_path;
if (FLAGS_save_result) {
// Get video information: resolution, fps
int video_width = static_cast<int>(capture.get(CV_CAP_PROP_FRAME_WIDTH));
int video_height =
static_cast<int>(capture.get(CV_CAP_PROP_FRAME_HEIGHT));
int video_fps = static_cast<int>(capture.get(CV_CAP_PROP_FPS));
int video_fourcc;
if (FLAGS_use_camera) {
video_fourcc = 828601953;
} else {
video_fourcc = static_cast<int>(capture.get(CV_CAP_PROP_FOURCC));
}
if (FLAGS_use_camera) {
time_t now = time(0);
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir,
std::to_string(now) + ".mp4");
} else {
video_out_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_video_path);
}
video_out.open(video_out_path.c_str(),
video_fourcc,
video_fps,
cv::Size(video_width, video_height),
true);
if (!video_out.isOpened()) {
std::cout << "Create video writer failed!" << std::endl;
return -1;
}
}
PaddleX::SegResult result;
cv::Mat frame;
int key;
while (capture.read(frame)) {
if (FLAGS_show_result || FLAGS_use_camera) {
key = cv::waitKey(1);
// When pressing `ESC`, then exit program and result video is saved
if (key == 27) {
break;
}
} else if (frame.empty()) {
break;
}
// Begin to predict
model.predict(frame, &result);
// Visualize results
std::vector<uint8_t> label_map(result.label_map.data.begin(),
result.label_map.data.end());
cv::Mat mask(result.label_map.shape[0],
result.label_map.shape[1],
CV_8UC1,
label_map.data());
int rows = result.label_map.shape[0];
int cols = result.label_map.shape[1];
cv::Mat vis_img = frame.clone();
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
int category_id = static_cast<int>(mask.at<uchar>(i, j));
if (category_id == 0) {
vis_img.at<cv::Vec3b>(i, j)[0] = 255;
vis_img.at<cv::Vec3b>(i, j)[1] = 255;
vis_img.at<cv::Vec3b>(i, j)[2] = 255;
}
}
}
if (FLAGS_show_result || FLAGS_use_camera) {
cv::imshow("human_seg", vis_img);
}
if (FLAGS_save_result) {
video_out.write(vis_img);
}
result.clear();
}
capture.release();
if (FLAGS_save_result) {
video_out.release();
std::cout << "Visualized output saved as " << video_out_path << std::endl;
}
if (FLAGS_show_result || FLAGS_use_camera) {
cv::destroyAllWindows();
}
} else {
PaddleX::SegResult result;
cv::Mat im = cv::imread(FLAGS_image, 1);
model.predict(im, &result);
// Visualize results
std::vector<uint8_t> label_map(result.label_map.data.begin(),
result.label_map.data.end());
cv::Mat mask(result.label_map.shape[0],
result.label_map.shape[1],
CV_8UC1,
label_map.data());
int rows = result.label_map.shape[0];
int cols = result.label_map.shape[1];
cv::Mat vis_img = im.clone();
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
int category_id = static_cast<int>(mask.at<uchar>(i, j));
if (category_id == 0) {
vis_img.at<cv::Vec3b>(i, j)[0] = 255;
vis_img.at<cv::Vec3b>(i, j)[1] = 255;
vis_img.at<cv::Vec3b>(i, j)[2] = 255;
}
}
}
std::string save_path =
PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_image);
cv::imwrite(save_path, vis_img);
result.clear();
std::cout << "Visualized output saved as " << save_path << std::endl;
}
return 0;
}
...@@ -148,8 +148,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -148,8 +148,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)| | use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)|
| gpu_id | GPU 设备ID, 默认值为0 | | gpu_id | GPU 设备ID, 默认值为0 |
| save_dir | 保存可视化结果的路径, 默认值为"output"| | save_dir | 保存可视化结果的路径, 默认值为"output"|
| det_key | 检测模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的检测模型 |
| seg_key | 分割模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的分割模型 |
| seg_batch_size | 分割的批量大小,默认为2 | | seg_batch_size | 分割的批量大小,默认为2 |
| thread_num | 分割预测的线程数,默认为cpu处理器个数 | | thread_num | 分割预测的线程数,默认为cpu处理器个数 |
| use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) | | use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) |
...@@ -163,13 +161,20 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -163,13 +161,20 @@ git clone https://github.com/PaddlePaddle/PaddleX
用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[导出inference模型](https://paddlex.readthedocs.io/zh_CN/latest/tutorials/deploy/deploy_server/deploy_python.html#inference)将模型导出为inference格式。 用于部署推理的模型应为inference格式,本案例提供的预训练模型均为inference格式,如若是重新训练的模型,需参考[导出inference模型](https://paddlex.readthedocs.io/zh_CN/latest/tutorials/deploy/deploy_server/deploy_python.html#inference)将模型导出为inference格式。
* 使用未加密的模型对单张图片做预测 * 使用未加密的模型对单张图片做预测
```shell ```shell
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image=\path\to\meter_test\20190822_168.jpg --use_gpu=1 --use_erode=1 --save_dir=output .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image=\path\to\meter_test\20190822_168.jpg --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
* 使用未加密的模型对图像列表做预测 * 使用未加密的模型对图像列表做预测
图像列表image_list.txt内容的格式如下,因绝对路径不同,暂未提供该文件,用户可根据实际情况自行生成:
```
\path\to\images\1.jpg
\path\to\images\2.jpg
...
\path\to\images\n.jpg
```
```shell ```shell
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image_list=\path\to\meter_test\image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --image_list=\path\to\meter_test\image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
...@@ -180,12 +185,12 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -180,12 +185,12 @@ git clone https://github.com/PaddlePaddle/PaddleX
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\det_inference_model --seg_model_dir=\path\to\seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
* 使用加密后的模型对单张图片做预测 * 使用加密后的模型对单张图片做预测
如果未对模型进行加密,请参考[加密PaddleX模型](../../docs/deploy/server/encryption.md#13-加密paddlex模型)对模型进行加密。例如加密后的检测模型所在目录为`\path\to\encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`\path\to\encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=` 如果未对模型进行加密,请参考[加密PaddleX模型](../../docs/deploy/server/encryption.md#13-加密paddlex模型)对模型进行加密。例如加密后的检测模型所在目录为`\path\to\encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`\path\to\encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=`
```shell ```shell
.\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\encrypted_det_inference_model --seg_model_dir=\path\to\encrypted_seg_inference_model --image=\path\to\test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY= .\paddlex_inference\meter_reader.exe --det_model_dir=\path\to\encrypted_det_inference_model --seg_model_dir=\path\to\encrypted_seg_inference_model --image=\path\to\test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=
``` ```
### Linux系统的jetson嵌入式设备安全部署 ### Linux系统的jetson嵌入式设备安全部署
...@@ -213,8 +218,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -213,8 +218,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
| use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)| | use_gpu | 是否使用 GPU 预测, 支持值为0或1(默认值为0)|
| gpu_id | GPU 设备ID, 默认值为0 | | gpu_id | GPU 设备ID, 默认值为0 |
| save_dir | 保存可视化结果的路径, 默认值为"output"| | save_dir | 保存可视化结果的路径, 默认值为"output"|
| det_key | 检测模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的检测模型 |
| seg_key | 分割模型加密过程中产生的密钥信息,默认值为""表示加载的是未加密的分割模型 |
| seg_batch_size | 分割的批量大小,默认为2 | | seg_batch_size | 分割的批量大小,默认为2 |
| thread_num | 分割预测的线程数,默认为cpu处理器个数 | | thread_num | 分割预测的线程数,默认为cpu处理器个数 |
| use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) | | use_camera | 是否使用摄像头采集图片,支持值为0或1(默认值为0) |
...@@ -234,6 +237,13 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -234,6 +237,13 @@ git clone https://github.com/PaddlePaddle/PaddleX
``` ```
* 使用未加密的模型对图像列表做预测 * 使用未加密的模型对图像列表做预测
图像列表image_list.txt内容的格式如下,因绝对路径不同,暂未提供该文件,用户可根据实际情况自行生成:
```
\path\to\images\1.jpg
\path\to\images\2.jpg
...
\path\to\images\n.jpg
```
```shell ```shell
./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --image_list=/path/to/image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output ./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --image_list=/path/to/image_list.txt --use_gpu=1 --use_erode=1 --save_dir=output
...@@ -245,15 +255,6 @@ git clone https://github.com/PaddlePaddle/PaddleX ...@@ -245,15 +255,6 @@ git clone https://github.com/PaddlePaddle/PaddleX
./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output ./build/meter_reader/meter_reader --det_model_dir=/path/to/det_inference_model --seg_model_dir=/path/to/seg_inference_model --use_camera=1 --use_gpu=1 --use_erode=1 --save_dir=output
``` ```
* 使用加密后的模型对单张图片做预测
如果未对模型进行加密,请参考[加密PaddleX模型](../../docs/deploy/server/encryption.md#13-加密paddlex模型)对模型进行加密。例如加密后的检测模型所在目录为`/path/to/encrypted_det_inference_model`,密钥为`yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0=`;加密后的分割模型所在目录为`/path/to/encrypted_seg_inference_model`,密钥为`DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=`
```shell
./build/meter_reader/meter_reader --det_model_dir=/path/to/encrypted_det_inference_model --seg_model_dir=/path/to/encrypted_seg_inference_model --image=/path/to/test.jpg --use_gpu=1 --use_erode=1 --save_dir=output --det_key yEBLDiBOdlj+5EsNNrABhfDuQGkdcreYcHcncqwdbx0= --seg_key DbVS64I9pFRo5XmQ8MNV2kSGsfEr4FKA6OH9OUhRrsY=
```
## <h2 id="5">模型训练</h2> ## <h2 id="5">模型训练</h2>
......
...@@ -51,7 +51,8 @@ DEFINE_string(seg_key, "", "Segmenter model key of encryption"); ...@@ -51,7 +51,8 @@ DEFINE_string(seg_key, "", "Segmenter model key of encryption");
DEFINE_string(image, "", "Path of test image file"); DEFINE_string(image, "", "Path of test image file");
DEFINE_string(image_list, "", "Path of test image list file"); DEFINE_string(image_list, "", "Path of test image list file");
DEFINE_string(save_dir, "output", "Path to save visualized image"); DEFINE_string(save_dir, "output", "Path to save visualized image");
DEFINE_double(score_threshold, 0.5, "Detected bbox whose score is lower than this threshlod is filtered"); DEFINE_double(score_threshold, 0.5,
"Detected bbox whose score is lower than this threshlod is filtered");
void predict(const cv::Mat &input_image, PaddleX::Model *det_model, void predict(const cv::Mat &input_image, PaddleX::Model *det_model,
PaddleX::Model *seg_model, const std::string save_dir, PaddleX::Model *seg_model, const std::string save_dir,
...@@ -207,7 +208,7 @@ int main(int argc, char **argv) { ...@@ -207,7 +208,7 @@ int main(int argc, char **argv) {
return -1; return -1;
} }
// 加载模型 // Load model
PaddleX::Model det_model; PaddleX::Model det_model;
det_model.Init(FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_use_trt, det_model.Init(FLAGS_det_model_dir, FLAGS_use_gpu, FLAGS_use_trt,
FLAGS_gpu_id, FLAGS_det_key); FLAGS_gpu_id, FLAGS_det_key);
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -15,6 +15,7 @@ ...@@ -15,6 +15,7 @@
from six import text_type as _text_type from six import text_type as _text_type
import argparse import argparse
import sys import sys
import os.path as osp
import paddlex.utils.logging as logging import paddlex.utils.logging as logging
...@@ -91,6 +92,33 @@ def arg_parser(): ...@@ -91,6 +92,33 @@ def arg_parser():
"-fs", "-fs",
default=None, default=None,
help="export inference model with fixed input shape:[w,h]") help="export inference model with fixed input shape:[w,h]")
parser.add_argument(
"--split_dataset",
"-sd",
action="store_true",
default=False,
help="split dataset with the split value")
parser.add_argument(
"--format",
"-f",
default=None,
help="define dataset format(ImageNet/COCO/VOC/Seg)")
parser.add_argument(
"--dataset_dir",
"-dd",
type=_text_type,
default=None,
help="define the path of dataset to be splited")
parser.add_argument(
"--val_value",
"-vv",
default=None,
help="define the value of validation dataset(E.g 0.2)")
parser.add_argument(
"--test_value",
"-tv",
default=None,
help="define the value of test dataset(E.g 0.1)")
return parser return parser
...@@ -159,6 +187,30 @@ def main(): ...@@ -159,6 +187,30 @@ def main():
pdx.tools.convert.dataset_conversion(args.source, args.to, args.pics, pdx.tools.convert.dataset_conversion(args.source, args.to, args.pics,
args.annotations, args.save_dir) args.annotations, args.save_dir)
if args.split_dataset:
assert args.dataset_dir is not None, "--dataset_dir should be defined while spliting dataset"
assert args.format is not None, "--form should be defined while spliting dataset"
assert args.val_value is not None, "--val_value should be defined while spliting dataset"
dataset_dir = args.dataset_dir
dataset_format = args.format.lower()
val_value = float(args.val_value)
test_value = float(args.test_value
if args.test_value is not None else 0)
save_dir = dataset_dir
if not dataset_format in ["coco", "imagenet", "voc", "seg"]:
logging.error(
"The dataset format is not correct defined.(support COCO/ImageNet/VOC/Seg)"
)
if not osp.exists(dataset_dir):
logging.error("The path of dataset to be splited doesn't exist.")
if val_value <= 0 or val_value >= 1 or test_value < 0 or test_value >= 1 or val_value + test_value >= 1:
logging.error("The value of split is not correct.")
pdx.tools.split.dataset_split(dataset_dir, dataset_format, val_value,
test_value, save_dir)
if __name__ == "__main__": if __name__ == "__main__":
main() main()
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -26,6 +26,7 @@ ResNet50 = models.ResNet50 ...@@ -26,6 +26,7 @@ ResNet50 = models.ResNet50
DarkNet53 = models.DarkNet53 DarkNet53 = models.DarkNet53
# detection # detection
YOLOv3 = models.YOLOv3 YOLOv3 = models.YOLOv3
PPYOLO = models.PPYOLO
#EAST = models.EAST #EAST = models.EAST
FasterRCNN = models.FasterRCNN FasterRCNN = models.FasterRCNN
MaskRCNN = models.MaskRCNN MaskRCNN = models.MaskRCNN
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -15,6 +15,8 @@ ...@@ -15,6 +15,8 @@
from __future__ import absolute_import from __future__ import absolute_import
import copy import copy
import os.path as osp import os.path as osp
import six
import sys
import random import random
import numpy as np import numpy as np
import paddlex.utils.logging as logging import paddlex.utils.logging as logging
...@@ -48,6 +50,12 @@ class CocoDetection(VOCDetection): ...@@ -48,6 +50,12 @@ class CocoDetection(VOCDetection):
shuffle=False): shuffle=False):
from pycocotools.coco import COCO from pycocotools.coco import COCO
try:
import shapely.ops
from shapely.geometry import Polygon, MultiPolygon, GeometryCollection
except:
six.reraise(*sys.exc_info())
super(VOCDetection, self).__init__( super(VOCDetection, self).__init__(
transforms=transforms, transforms=transforms,
num_workers=num_workers, num_workers=num_workers,
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -115,7 +115,7 @@ def multithread_reader(mapper, ...@@ -115,7 +115,7 @@ def multithread_reader(mapper,
while not isinstance(sample, EndSignal): while not isinstance(sample, EndSignal):
batch_data.append(sample) batch_data.append(sample)
if len(batch_data) == batch_size: if len(batch_data) == batch_size:
batch_data = generate_minibatch(batch_data) batch_data = generate_minibatch(batch_data, mapper=mapper)
yield batch_data yield batch_data
batch_data = [] batch_data = []
sample = out_queue.get() sample = out_queue.get()
...@@ -127,11 +127,11 @@ def multithread_reader(mapper, ...@@ -127,11 +127,11 @@ def multithread_reader(mapper,
else: else:
batch_data.append(sample) batch_data.append(sample)
if len(batch_data) == batch_size: if len(batch_data) == batch_size:
batch_data = generate_minibatch(batch_data) batch_data = generate_minibatch(batch_data, mapper=mapper)
yield batch_data yield batch_data
batch_data = [] batch_data = []
if not drop_last and len(batch_data) != 0: if not drop_last and len(batch_data) != 0:
batch_data = generate_minibatch(batch_data) batch_data = generate_minibatch(batch_data, mapper=mapper)
yield batch_data yield batch_data
batch_data = [] batch_data = []
...@@ -188,18 +188,21 @@ def multiprocess_reader(mapper, ...@@ -188,18 +188,21 @@ def multiprocess_reader(mapper,
else: else:
batch_data.append(sample) batch_data.append(sample)
if len(batch_data) == batch_size: if len(batch_data) == batch_size:
batch_data = generate_minibatch(batch_data) batch_data = generate_minibatch(batch_data, mapper=mapper)
yield batch_data yield batch_data
batch_data = [] batch_data = []
if len(batch_data) != 0 and not drop_last: if len(batch_data) != 0 and not drop_last:
batch_data = generate_minibatch(batch_data) batch_data = generate_minibatch(batch_data, mapper=mapper)
yield batch_data yield batch_data
batch_data = [] batch_data = []
return queue_reader return queue_reader
def generate_minibatch(batch_data, label_padding_value=255): def generate_minibatch(batch_data, label_padding_value=255, mapper=None):
if mapper is not None and mapper.batch_transforms is not None:
for op in mapper.batch_transforms:
batch_data = op(batch_data)
# if batch_size is 1, do not pad the image # if batch_size is 1, do not pad the image
if len(batch_data) == 1: if len(batch_data) == 1:
return batch_data return batch_data
...@@ -218,14 +221,13 @@ def generate_minibatch(batch_data, label_padding_value=255): ...@@ -218,14 +221,13 @@ def generate_minibatch(batch_data, label_padding_value=255):
(im_c, max_shape[1], max_shape[2]), dtype=np.float32) (im_c, max_shape[1], max_shape[2]), dtype=np.float32)
padding_im[:, :im_h, :im_w] = data[0] padding_im[:, :im_h, :im_w] = data[0]
if len(data) > 2: if len(data) > 2:
# padding the image, label and insert 'padding' into `im_info` of segmentation during evaluating phase. # padding the image, label and insert 'padding' into `im_info` of segmentation during evaluating phase.
if len(data[1]) == 0 or 'padding' not in [ if len(data[1]) == 0 or 'padding' not in [
data[1][i][0] for i in range(len(data[1])) data[1][i][0] for i in range(len(data[1]))
]: ]:
data[1].append(('padding', [im_h, im_w])) data[1].append(('padding', [im_h, im_w]))
padding_batch.append((padding_im, data[1], data[2])) padding_batch.append((padding_im, data[1], data[2]))
elif len(data) > 1: elif len(data) > 1:
if isinstance(data[1], np.ndarray) and len(data[1].shape) > 1: if isinstance(data[1], np.ndarray) and len(data[1].shape) > 1:
# padding the image and label of segmentation during the training # padding the image and label of segmentation during the training
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -25,6 +25,7 @@ from .voc import VOCDetection ...@@ -25,6 +25,7 @@ from .voc import VOCDetection
from .dataset import is_pic from .dataset import is_pic
from .dataset import get_encoding from .dataset import get_encoding
class EasyDataDet(VOCDetection): class EasyDataDet(VOCDetection):
"""读取EasyDataDet格式的检测数据集,并对样本进行相应的处理。 """读取EasyDataDet格式的检测数据集,并对样本进行相应的处理。
...@@ -41,7 +42,7 @@ class EasyDataDet(VOCDetection): ...@@ -41,7 +42,7 @@ class EasyDataDet(VOCDetection):
线程和'process'进程两种方式。默认为'process'(Windows和Mac下会强制使用thread,该参数无效)。 线程和'process'进程两种方式。默认为'process'(Windows和Mac下会强制使用thread,该参数无效)。
shuffle (bool): 是否需要对数据集中样本打乱顺序。默认为False。 shuffle (bool): 是否需要对数据集中样本打乱顺序。默认为False。
""" """
def __init__(self, def __init__(self,
data_dir, data_dir,
file_list, file_list,
...@@ -60,12 +61,12 @@ class EasyDataDet(VOCDetection): ...@@ -60,12 +61,12 @@ class EasyDataDet(VOCDetection):
self.file_list = list() self.file_list = list()
self.labels = list() self.labels = list()
self._epoch = 0 self._epoch = 0
annotations = {} annotations = {}
annotations['images'] = [] annotations['images'] = []
annotations['categories'] = [] annotations['categories'] = []
annotations['annotations'] = [] annotations['annotations'] = []
cname2cid = {} cname2cid = {}
label_id = 1 label_id = 1
with open(label_list, encoding=get_encoding(label_list)) as fr: with open(label_list, encoding=get_encoding(label_list)) as fr:
...@@ -80,7 +81,7 @@ class EasyDataDet(VOCDetection): ...@@ -80,7 +81,7 @@ class EasyDataDet(VOCDetection):
'id': v, 'id': v,
'name': k 'name': k
}) })
from pycocotools.mask import decode from pycocotools.mask import decode
ct = 0 ct = 0
ann_ct = 0 ann_ct = 0
...@@ -95,8 +96,8 @@ class EasyDataDet(VOCDetection): ...@@ -95,8 +96,8 @@ class EasyDataDet(VOCDetection):
if not osp.isfile(json_file): if not osp.isfile(json_file):
continue continue
if not osp.exists(img_file): if not osp.exists(img_file):
raise IOError( raise IOError('The image file {} is not exist!'.format(
'The image file {} is not exist!'.format(img_file)) img_file))
with open(json_file, mode='r', \ with open(json_file, mode='r', \
encoding=get_encoding(json_file)) as j: encoding=get_encoding(json_file)) as j:
json_info = json.load(j) json_info = json.load(j)
...@@ -127,21 +128,15 @@ class EasyDataDet(VOCDetection): ...@@ -127,21 +128,15 @@ class EasyDataDet(VOCDetection):
mask = decode(mask_dict) mask = decode(mask_dict)
gt_poly[i] = self.mask2polygon(mask) gt_poly[i] = self.mask2polygon(mask)
annotations['annotations'].append({ annotations['annotations'].append({
'iscrowd': 'iscrowd': 0,
0, 'image_id': int(im_id[0]),
'image_id':
int(im_id[0]),
'bbox': [x1, y1, x2 - x1 + 1, y2 - y1 + 1], 'bbox': [x1, y1, x2 - x1 + 1, y2 - y1 + 1],
'area': 'area': float((x2 - x1 + 1) * (y2 - y1 + 1)),
float((x2 - x1 + 1) * (y2 - y1 + 1)), 'segmentation': [[x1, y1, x1, y2, x2, y2, x2, y1]]
'segmentation': if gt_poly[i] is None else gt_poly[i],
[[x1, y1, x1, y2, x2, y2, x2, y1]] if gt_poly[i] is None else gt_poly[i], 'category_id': cname2cid[cname],
'category_id': 'id': ann_ct,
cname2cid[cname], 'difficult': 0
'id':
ann_ct,
'difficult':
0
}) })
ann_ct += 1 ann_ct += 1
im_info = { im_info = {
...@@ -162,14 +157,10 @@ class EasyDataDet(VOCDetection): ...@@ -162,14 +157,10 @@ class EasyDataDet(VOCDetection):
self.file_list.append([img_file, voc_rec]) self.file_list.append([img_file, voc_rec])
ct += 1 ct += 1
annotations['images'].append({ annotations['images'].append({
'height': 'height': im_h,
im_h, 'width': im_w,
'width': 'id': int(im_id[0]),
im_w, 'file_name': osp.split(img_file)[1]
'id':
int(im_id[0]),
'file_name':
osp.split(img_file)[1]
}) })
if not len(self.file_list) > 0: if not len(self.file_list) > 0:
...@@ -181,13 +172,13 @@ class EasyDataDet(VOCDetection): ...@@ -181,13 +172,13 @@ class EasyDataDet(VOCDetection):
self.coco_gt = COCO() self.coco_gt = COCO()
self.coco_gt.dataset = annotations self.coco_gt.dataset = annotations
self.coco_gt.createIndex() self.coco_gt.createIndex()
def mask2polygon(self, mask): def mask2polygon(self, mask):
contours, hierarchy = cv2.findContours( contours, hierarchy = cv2.findContours(
(mask).astype(np.uint8), cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE) (mask).astype(np.uint8), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
segmentation = [] segmentation = []
for contour in contours: for contour in contours:
contour_list = contour.flatten().tolist() contour_list = contour.flatten().tolist()
if len(contour_list) > 4: if len(contour_list) > 4:
segmentation.append(contour_list) segmentation.append(contour_list)
return segmentation return segmentation
\ No newline at end of file
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -25,6 +25,7 @@ from .dataset import Dataset ...@@ -25,6 +25,7 @@ from .dataset import Dataset
from .dataset import get_encoding from .dataset import get_encoding
from .dataset import is_pic from .dataset import is_pic
class EasyDataSeg(Dataset): class EasyDataSeg(Dataset):
"""读取EasyDataSeg语义分割任务数据集,并对样本进行相应的处理。 """读取EasyDataSeg语义分割任务数据集,并对样本进行相应的处理。
...@@ -67,7 +68,7 @@ class EasyDataSeg(Dataset): ...@@ -67,7 +68,7 @@ class EasyDataSeg(Dataset):
cname2cid[line.strip()] = label_id cname2cid[line.strip()] = label_id
label_id += 1 label_id += 1
self.labels.append(line.strip()) self.labels.append(line.strip())
with open(file_list, encoding=get_encoding(file_list)) as f: with open(file_list, encoding=get_encoding(file_list)) as f:
for line in f: for line in f:
img_file, json_file = [osp.join(data_dir, x) \ img_file, json_file = [osp.join(data_dir, x) \
...@@ -79,8 +80,8 @@ class EasyDataSeg(Dataset): ...@@ -79,8 +80,8 @@ class EasyDataSeg(Dataset):
if not osp.isfile(json_file): if not osp.isfile(json_file):
continue continue
if not osp.exists(img_file): if not osp.exists(img_file):
raise IOError( raise IOError('The image file {} is not exist!'.format(
'The image file {} is not exist!'.format(img_file)) img_file))
with open(json_file, mode='r', \ with open(json_file, mode='r', \
encoding=get_encoding(json_file)) as j: encoding=get_encoding(json_file)) as j:
json_info = json.load(j) json_info = json.load(j)
...@@ -97,7 +98,8 @@ class EasyDataSeg(Dataset): ...@@ -97,7 +98,8 @@ class EasyDataSeg(Dataset):
mask_dict['counts'] = obj['mask'].encode() mask_dict['counts'] = obj['mask'].encode()
mask = decode(mask_dict) mask = decode(mask_dict)
mask *= cid mask *= cid
conflict_index = np.where(((lable_npy > 0) & (mask == cid)) == True) conflict_index = np.where(((lable_npy > 0) &
(mask == cid)) == True)
mask[conflict_index] = 0 mask[conflict_index] = 0
lable_npy += mask lable_npy += mask
self.file_list.append([img_file, lable_npy]) self.file_list.append([img_file, lable_npy])
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -278,8 +278,8 @@ class PageAllocator(object): ...@@ -278,8 +278,8 @@ class PageAllocator(object):
def set_alloc_info(self, alloc_pos, used_pages): def set_alloc_info(self, alloc_pos, used_pages):
""" set allocating position to new value """ set allocating position to new value
""" """
memcopy(self._base[4:12], struct.pack( memcopy(self._base[4:12],
str('II'), alloc_pos, used_pages)) struct.pack(str('II'), alloc_pos, used_pages))
def set_page_status(self, start, page_num, status): def set_page_status(self, start, page_num, status):
""" set pages from 'start' to 'end' with new same status 'status' """ set pages from 'start' to 'end' with new same status 'status'
...@@ -525,8 +525,8 @@ class SharedMemoryMgr(object): ...@@ -525,8 +525,8 @@ class SharedMemoryMgr(object):
logger.info('destroy [%s]' % (self)) logger.info('destroy [%s]' % (self))
if not self._released and not self._allocator.empty(): if not self._released and not self._allocator.empty():
logger.debug( logger.debug('not empty when delete this SharedMemoryMgr[%s]' %
'not empty when delete this SharedMemoryMgr[%s]' % (self)) (self))
else: else:
self._released = True self._released = True
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -38,6 +38,7 @@ from .classifier import HRNet_W18 ...@@ -38,6 +38,7 @@ from .classifier import HRNet_W18
from .classifier import AlexNet from .classifier import AlexNet
from .base import BaseAPI from .base import BaseAPI
from .yolo_v3 import YOLOv3 from .yolo_v3 import YOLOv3
from .ppyolo import PPYOLO
from .faster_rcnn import FasterRCNN from .faster_rcnn import FasterRCNN
from .mask_rcnn import MaskRCNN from .mask_rcnn import MaskRCNN
from .unet import UNet from .unet import UNet
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -246,8 +246,8 @@ class BaseAPI: ...@@ -246,8 +246,8 @@ class BaseAPI:
logging.info( logging.info(
"Load pretrain weights from {}.".format(pretrain_weights), "Load pretrain weights from {}.".format(pretrain_weights),
use_color=True) use_color=True)
paddlex.utils.utils.load_pretrain_weights(self.exe, self.train_prog, paddlex.utils.utils.load_pretrain_weights(
pretrain_weights, fuse_bn) self.exe, self.train_prog, pretrain_weights, fuse_bn)
# 进行裁剪 # 进行裁剪
if sensitivities_file is not None: if sensitivities_file is not None:
import paddleslim import paddleslim
...@@ -351,7 +351,9 @@ class BaseAPI: ...@@ -351,7 +351,9 @@ class BaseAPI:
logging.info("Model saved in {}.".format(save_dir)) logging.info("Model saved in {}.".format(save_dir))
def export_inference_model(self, save_dir): def export_inference_model(self, save_dir):
test_input_names = [var.name for var in list(self.test_inputs.values())] test_input_names = [
var.name for var in list(self.test_inputs.values())
]
test_outputs = list(self.test_outputs.values()) test_outputs = list(self.test_outputs.values())
with fluid.scope_guard(self.scope): with fluid.scope_guard(self.scope):
if self.__class__.__name__ == 'MaskRCNN': if self.__class__.__name__ == 'MaskRCNN':
...@@ -389,7 +391,8 @@ class BaseAPI: ...@@ -389,7 +391,8 @@ class BaseAPI:
# 模型保存成功的标志 # 模型保存成功的标志
open(osp.join(save_dir, '.success'), 'w').close() open(osp.join(save_dir, '.success'), 'w').close()
logging.info("Model for inference deploy saved in {}.".format(save_dir)) logging.info("Model for inference deploy saved in {}.".format(
save_dir))
def train_loop(self, def train_loop(self,
num_epochs, num_epochs,
...@@ -516,11 +519,13 @@ class BaseAPI: ...@@ -516,11 +519,13 @@ class BaseAPI:
eta = ((num_epochs - i) * total_num_steps - step - 1 eta = ((num_epochs - i) * total_num_steps - step - 1
) * avg_step_time ) * avg_step_time
if time_eval_one_epoch is not None: if time_eval_one_epoch is not None:
eval_eta = (total_eval_times - i // save_interval_epochs eval_eta = (
) * time_eval_one_epoch total_eval_times - i // save_interval_epochs
) * time_eval_one_epoch
else: else:
eval_eta = (total_eval_times - i // save_interval_epochs eval_eta = (
) * total_num_steps_eval * avg_step_time total_eval_times - i // save_interval_epochs
) * total_num_steps_eval * avg_step_time
eta_str = seconds_to_hms(eta + eval_eta) eta_str = seconds_to_hms(eta + eval_eta)
logging.info( logging.info(
...@@ -543,6 +548,8 @@ class BaseAPI: ...@@ -543,6 +548,8 @@ class BaseAPI:
current_save_dir = osp.join(save_dir, "epoch_{}".format(i + 1)) current_save_dir = osp.join(save_dir, "epoch_{}".format(i + 1))
if not osp.isdir(current_save_dir): if not osp.isdir(current_save_dir):
os.makedirs(current_save_dir) os.makedirs(current_save_dir)
if getattr(self, 'use_ema', False):
self.exe.run(self.ema.apply_program)
if eval_dataset is not None and eval_dataset.num_samples > 0: if eval_dataset is not None and eval_dataset.num_samples > 0:
self.eval_metrics, self.eval_details = self.evaluate( self.eval_metrics, self.eval_details = self.evaluate(
eval_dataset=eval_dataset, eval_dataset=eval_dataset,
...@@ -569,6 +576,8 @@ class BaseAPI: ...@@ -569,6 +576,8 @@ class BaseAPI:
log_writer.add_scalar( log_writer.add_scalar(
"Metrics/Eval(Epoch): {}".format(k), v, i + 1) "Metrics/Eval(Epoch): {}".format(k), v, i + 1)
self.save_model(save_dir=current_save_dir) self.save_model(save_dir=current_save_dir)
if getattr(self, 'use_ema', False):
self.exe.run(self.ema.restore_program)
time_eval_one_epoch = time.time() - eval_epoch_start_time time_eval_one_epoch = time.time() - eval_epoch_start_time
eval_epoch_start_time = time.time() eval_epoch_start_time = time.time()
if best_model_epoch > 0: if best_model_epoch > 0:
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -37,7 +37,7 @@ class DeepLabv3p(BaseAPI): ...@@ -37,7 +37,7 @@ class DeepLabv3p(BaseAPI):
num_classes (int): 类别数。 num_classes (int): 类别数。
backbone (str): DeepLabv3+的backbone网络,实现特征图的计算,取值范围为['Xception65', 'Xception41', backbone (str): DeepLabv3+的backbone网络,实现特征图的计算,取值范围为['Xception65', 'Xception41',
'MobileNetV2_x0.25', 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5',
'MobileNetV2_x2.0']。默认'MobileNetV2_x1.0'。 'MobileNetV2_x2.0', 'MobileNetV3_large_x1_0_ssld']。默认'MobileNetV2_x1.0'。
output_stride (int): backbone 输出特征图相对于输入的下采样倍数,一般取值为8或16。默认16。 output_stride (int): backbone 输出特征图相对于输入的下采样倍数,一般取值为8或16。默认16。
aspp_with_sep_conv (bool): 在asspp模块是否采用separable convolutions。默认True。 aspp_with_sep_conv (bool): 在asspp模块是否采用separable convolutions。默认True。
decoder_use_sep_conv (bool): decoder模块是否采用separable convolutions。默认True。 decoder_use_sep_conv (bool): decoder模块是否采用separable convolutions。默认True。
...@@ -51,10 +51,13 @@ class DeepLabv3p(BaseAPI): ...@@ -51,10 +51,13 @@ class DeepLabv3p(BaseAPI):
自行计算相应的权重,每一类的权重为:每类的比例 * num_classes。class_weight取默认值None时,各类的权重1, 自行计算相应的权重,每一类的权重为:每类的比例 * num_classes。class_weight取默认值None时,各类的权重1,
即平时使用的交叉熵损失函数。 即平时使用的交叉熵损失函数。
ignore_index (int): label上忽略的值,label为ignore_index的像素不参与损失函数的计算。默认255。 ignore_index (int): label上忽略的值,label为ignore_index的像素不参与损失函数的计算。默认255。
pooling_crop_size (list): 当backbone为MobileNetV3_large_x1_0_ssld时,需设置为训练过程中模型输入大小, 格式为[W, H]。
在encoder模块中获取图像平均值时被用到,若为None,则直接求平均值;若为模型输入大小,则使用'pool'算子得到平均值。
默认值为None。
Raises: Raises:
ValueError: use_bce_loss或use_dice_loss为真且num_calsses > 2。 ValueError: use_bce_loss或use_dice_loss为真且num_calsses > 2。
ValueError: backbone取值不在['Xception65', 'Xception41', 'MobileNetV2_x0.25', ValueError: backbone取值不在['Xception65', 'Xception41', 'MobileNetV2_x0.25',
'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0']之内。 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0', 'MobileNetV3_large_x1_0_ssld']之内。
ValueError: class_weight为list, 但长度不等于num_class。 ValueError: class_weight为list, 但长度不等于num_class。
class_weight为str, 但class_weight.low()不等于dynamic。 class_weight为str, 但class_weight.low()不等于dynamic。
TypeError: class_weight不为None时,其类型不是list或str。 TypeError: class_weight不为None时,其类型不是list或str。
...@@ -71,7 +74,8 @@ class DeepLabv3p(BaseAPI): ...@@ -71,7 +74,8 @@ class DeepLabv3p(BaseAPI):
use_bce_loss=False, use_bce_loss=False,
use_dice_loss=False, use_dice_loss=False,
class_weight=None, class_weight=None,
ignore_index=255): ignore_index=255,
pooling_crop_size=None):
self.init_params = locals() self.init_params = locals()
super(DeepLabv3p, self).__init__('segmenter') super(DeepLabv3p, self).__init__('segmenter')
# dice_loss或bce_loss只适用两类分割中 # dice_loss或bce_loss只适用两类分割中
...@@ -85,12 +89,12 @@ class DeepLabv3p(BaseAPI): ...@@ -85,12 +89,12 @@ class DeepLabv3p(BaseAPI):
if backbone not in [ if backbone not in [
'Xception65', 'Xception41', 'MobileNetV2_x0.25', 'Xception65', 'Xception41', 'MobileNetV2_x0.25',
'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x0.5', 'MobileNetV2_x1.0', 'MobileNetV2_x1.5',
'MobileNetV2_x2.0' 'MobileNetV2_x2.0', 'MobileNetV3_large_x1_0_ssld'
]: ]:
raise ValueError( raise ValueError(
"backbone: {} is set wrong. it should be one of " "backbone: {} is set wrong. it should be one of "
"('Xception65', 'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5'," "('Xception65', 'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5',"
" 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0')". " 'MobileNetV2_x1.0', 'MobileNetV2_x1.5', 'MobileNetV2_x2.0', 'MobileNetV3_large_x1_0_ssld')".
format(backbone)) format(backbone))
if class_weight is not None: if class_weight is not None:
...@@ -121,6 +125,30 @@ class DeepLabv3p(BaseAPI): ...@@ -121,6 +125,30 @@ class DeepLabv3p(BaseAPI):
self.labels = None self.labels = None
self.sync_bn = True self.sync_bn = True
self.fixed_input_shape = None self.fixed_input_shape = None
self.pooling_stride = [1, 1]
self.pooling_crop_size = pooling_crop_size
self.aspp_with_se = False
self.se_use_qsigmoid = False
self.aspp_convs_filters = 256
self.aspp_with_concat_projection = True
self.add_image_level_feature = True
self.use_sum_merge = False
self.conv_filters = 256
self.output_is_logits = False
self.backbone_lr_mult_list = None
if 'MobileNetV3' in backbone:
self.output_stride = 32
self.pooling_stride = (4, 5)
self.aspp_with_se = True
self.se_use_qsigmoid = True
self.aspp_convs_filters = 128
self.aspp_with_concat_projection = False
self.add_image_level_feature = False
self.use_sum_merge = True
self.output_is_logits = True
if self.output_is_logits:
self.conv_filters = self.num_classes
self.backbone_lr_mult_list = [0.15, 0.35, 0.65, 0.85, 1]
def _get_backbone(self, backbone): def _get_backbone(self, backbone):
def mobilenetv2(backbone): def mobilenetv2(backbone):
...@@ -167,10 +195,22 @@ class DeepLabv3p(BaseAPI): ...@@ -167,10 +195,22 @@ class DeepLabv3p(BaseAPI):
end_points=end_points, end_points=end_points,
decode_points=decode_points) decode_points=decode_points)
def mobilenetv3(backbone):
scale = 1.0
lr_mult_list = self.backbone_lr_mult_list
return paddlex.cv.nets.MobileNetV3(
scale=scale,
model_name='large',
output_stride=self.output_stride,
lr_mult_list=lr_mult_list,
for_seg=True)
if 'Xception' in backbone: if 'Xception' in backbone:
return xception(backbone) return xception(backbone)
elif 'MobileNetV2' in backbone: elif 'MobileNetV2' in backbone:
return mobilenetv2(backbone) return mobilenetv2(backbone)
elif 'MobileNetV3' in backbone:
return mobilenetv3(backbone)
def build_net(self, mode='train'): def build_net(self, mode='train'):
model = paddlex.cv.nets.segmentation.DeepLabv3p( model = paddlex.cv.nets.segmentation.DeepLabv3p(
...@@ -186,7 +226,17 @@ class DeepLabv3p(BaseAPI): ...@@ -186,7 +226,17 @@ class DeepLabv3p(BaseAPI):
use_dice_loss=self.use_dice_loss, use_dice_loss=self.use_dice_loss,
class_weight=self.class_weight, class_weight=self.class_weight,
ignore_index=self.ignore_index, ignore_index=self.ignore_index,
fixed_input_shape=self.fixed_input_shape) fixed_input_shape=self.fixed_input_shape,
pooling_stride=self.pooling_stride,
pooling_crop_size=self.pooling_crop_size,
aspp_with_se=self.aspp_with_se,
se_use_qsigmoid=self.se_use_qsigmoid,
aspp_convs_filters=self.aspp_convs_filters,
aspp_with_concat_projection=self.aspp_with_concat_projection,
add_image_level_feature=self.add_image_level_feature,
use_sum_merge=self.use_sum_merge,
conv_filters=self.conv_filters,
output_is_logits=self.output_is_logits)
inputs = model.generate_inputs() inputs = model.generate_inputs()
model_out = model.build_net(inputs) model_out = model.build_net(inputs)
outputs = OrderedDict() outputs = OrderedDict()
...@@ -360,18 +410,16 @@ class DeepLabv3p(BaseAPI): ...@@ -360,18 +410,16 @@ class DeepLabv3p(BaseAPI):
pred = pred[0:num_samples] pred = pred[0:num_samples]
for i in range(num_samples): for i in range(num_samples):
one_pred = pred[i].astype('uint8') one_pred = np.squeeze(pred[i]).astype('uint8')
one_label = labels[i] one_label = labels[i]
for info in im_info[i][::-1]: for info in im_info[i][::-1]:
if info[0] == 'resize': if info[0] == 'resize':
w, h = info[1][1], info[1][0] w, h = info[1][1], info[1][0]
one_pred = cv2.resize(one_pred, (w, h), cv2.INTER_NEAREST) one_pred = cv2.resize(one_pred, (w, h),
cv2.INTER_NEAREST)
elif info[0] == 'padding': elif info[0] == 'padding':
w, h = info[1][1], info[1][0] w, h = info[1][1], info[1][0]
one_pred = one_pred[0:h, 0:w] one_pred = one_pred[0:h, 0:w]
else:
raise Exception("Unexpected info '{}' in im_info".format(
info[0]))
one_pred = one_pred.astype('int64') one_pred = one_pred.astype('int64')
one_pred = one_pred[np.newaxis, :, :, np.newaxis] one_pred = one_pred[np.newaxis, :, :, np.newaxis]
one_label = one_label[np.newaxis, np.newaxis, :, :] one_label = one_label[np.newaxis, np.newaxis, :, :]
...@@ -429,9 +477,6 @@ class DeepLabv3p(BaseAPI): ...@@ -429,9 +477,6 @@ class DeepLabv3p(BaseAPI):
w, h = info[1][1], info[1][0] w, h = info[1][1], info[1][0]
pred = pred[0:h, 0:w] pred = pred[0:h, 0:w]
logit = logit[0:h, 0:w, :] logit = logit[0:h, 0:w, :]
else:
raise Exception("Unexpected info '{}' in im_info".format(
info[0]))
pred_list.append(pred) pred_list.append(pred)
logit_list.append(logit) logit_list.append(logit)
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -103,8 +103,8 @@ def load_model(model_dir, fixed_input_shape=None): ...@@ -103,8 +103,8 @@ def load_model(model_dir, fixed_input_shape=None):
model.model_type, info['Transforms'], info['BatchTransforms']) model.model_type, info['Transforms'], info['BatchTransforms'])
model.eval_transforms = copy.deepcopy(model.test_transforms) model.eval_transforms = copy.deepcopy(model.test_transforms)
else: else:
model.test_transforms = build_transforms(model.model_type, model.test_transforms = build_transforms(
info['Transforms'], to_rgb) model.model_type, info['Transforms'], to_rgb)
model.eval_transforms = copy.deepcopy(model.test_transforms) model.eval_transforms = copy.deepcopy(model.test_transforms)
if '_Attributes' in info: if '_Attributes' in info:
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -280,8 +280,9 @@ class MaskRCNN(FasterRCNN): ...@@ -280,8 +280,9 @@ class MaskRCNN(FasterRCNN):
total_steps = math.ceil(eval_dataset.num_samples * 1.0 / batch_size) total_steps = math.ceil(eval_dataset.num_samples * 1.0 / batch_size)
results = list() results = list()
logging.info("Start to evaluating(total_samples={}, total_steps={})...". logging.info(
format(eval_dataset.num_samples, total_steps)) "Start to evaluating(total_samples={}, total_steps={})...".format(
eval_dataset.num_samples, total_steps))
for step, data in tqdm.tqdm( for step, data in tqdm.tqdm(
enumerate(data_generator()), total=total_steps): enumerate(data_generator()), total=total_steps):
images = np.array([d[0] for d in data]).astype('float32') images = np.array([d[0] for d in data]).astype('float32')
...@@ -325,7 +326,8 @@ class MaskRCNN(FasterRCNN): ...@@ -325,7 +326,8 @@ class MaskRCNN(FasterRCNN):
zip(['bbox_map', 'segm_map'], zip(['bbox_map', 'segm_map'],
[ap_stats[0][1], ap_stats[1][1]])) [ap_stats[0][1], ap_stats[1][1]]))
else: else:
metrics = OrderedDict(zip(['bbox_map', 'segm_map'], [0.0, 0.0])) metrics = OrderedDict(
zip(['bbox_map', 'segm_map'], [0.0, 0.0]))
elif metric == 'COCO': elif metric == 'COCO':
if isinstance(ap_stats[0], np.ndarray) and isinstance(ap_stats[1], if isinstance(ap_stats[0], np.ndarray) and isinstance(ap_stats[1],
np.ndarray): np.ndarray):
...@@ -429,8 +431,8 @@ class MaskRCNN(FasterRCNN): ...@@ -429,8 +431,8 @@ class MaskRCNN(FasterRCNN):
if transforms is None: if transforms is None:
transforms = self.test_transforms transforms = self.test_transforms
im, im_resize_info, im_shape = FasterRCNN._preprocess( im, im_resize_info, im_shape = FasterRCNN._preprocess(
img_file_list, transforms, self.model_type, self.__class__.__name__, img_file_list, transforms, self.model_type,
thread_num) self.__class__.__name__, thread_num)
with fluid.scope_guard(self.scope): with fluid.scope_guard(self.scope):
result = self.exe.run(self.test_prog, result = self.exe.run(self.test_prog,
......
此差异已折叠。
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -80,7 +80,9 @@ class PaddleXPostTrainingQuantization(PostTrainingQuantization): ...@@ -80,7 +80,9 @@ class PaddleXPostTrainingQuantization(PostTrainingQuantization):
self._support_activation_quantize_type = [ self._support_activation_quantize_type = [
'range_abs_max', 'moving_average_abs_max', 'abs_max' 'range_abs_max', 'moving_average_abs_max', 'abs_max'
] ]
self._support_weight_quantize_type = ['abs_max', 'channel_wise_abs_max'] self._support_weight_quantize_type = [
'abs_max', 'channel_wise_abs_max'
]
self._support_algo_type = ['KL', 'abs_max', 'min_max'] self._support_algo_type = ['KL', 'abs_max', 'min_max']
self._support_quantize_op_type = \ self._support_quantize_op_type = \
list(set(QuantizationTransformPass._supported_quantizable_op_type + list(set(QuantizationTransformPass._supported_quantizable_op_type +
...@@ -240,8 +242,8 @@ class PaddleXPostTrainingQuantization(PostTrainingQuantization): ...@@ -240,8 +242,8 @@ class PaddleXPostTrainingQuantization(PostTrainingQuantization):
'[Calculate weight] Weight_id={}/{}, time_each_weight={} s.'. '[Calculate weight] Weight_id={}/{}, time_each_weight={} s.'.
format( format(
str(ct), str(ct),
str(len(self._quantized_weight_var_name)), str(end - str(len(self._quantized_weight_var_name)),
start))) str(end - start)))
ct += 1 ct += 1
ct = 1 ct = 1
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -288,8 +288,8 @@ def get_params_ratios(sensitivities_file, eval_metric_loss=0.05): ...@@ -288,8 +288,8 @@ def get_params_ratios(sensitivities_file, eval_metric_loss=0.05):
if not osp.exists(sensitivities_file): if not osp.exists(sensitivities_file):
raise Exception('The sensitivities file is not exists!') raise Exception('The sensitivities file is not exists!')
sensitivitives = paddleslim.prune.load_sensitivities(sensitivities_file) sensitivitives = paddleslim.prune.load_sensitivities(sensitivities_file)
params_ratios = paddleslim.prune.get_ratios_by_loss( params_ratios = paddleslim.prune.get_ratios_by_loss(sensitivitives,
sensitivitives, eval_metric_loss) eval_metric_loss)
return params_ratios return params_ratios
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -47,8 +47,7 @@ def visualize(model, sensitivities_file, save_dir='./'): ...@@ -47,8 +47,7 @@ def visualize(model, sensitivities_file, save_dir='./'):
y.append(loss_thresh) y.append(loss_thresh)
plt.plot(x, y, color='green', linewidth=0.5, marker='o', markersize=3) plt.plot(x, y, color='green', linewidth=0.5, marker='o', markersize=3)
my_x_ticks = np.arange( my_x_ticks = np.arange(
min(np.array(x)) - 0.01, min(np.array(x)) - 0.01, max(np.array(x)) + 0.01, 0.05)
max(np.array(x)) + 0.01, 0.05)
my_y_ticks = np.arange(0.05, 1, 0.05) my_y_ticks = np.arange(0.05, 1, 0.05)
plt.xticks(my_x_ticks, rotation=15, fontsize=8) plt.xticks(my_x_ticks, rotation=15, fontsize=8)
plt.yticks(my_y_ticks, fontsize=8) plt.yticks(my_y_ticks, fontsize=8)
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
# You may obtain a copy of the License at # You may obtain a copy of the License at
# #
# http://www.apache.org/licenses/LICENSE-2.0 # http://www.apache.org/licenses/LICENSE-2.0
# #
# Unless required by applicable law or agreed to in writing, software # Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, # distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
# You may obtain a copy of the License at # You may obtain a copy of the License at
# #
# http://www.apache.org/licenses/LICENSE-2.0 # http://www.apache.org/licenses/LICENSE-2.0
# #
# Unless required by applicable law or agreed to in writing, software # Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, # distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
...@@ -158,8 +158,8 @@ def loadRes(coco_obj, anns): ...@@ -158,8 +158,8 @@ def loadRes(coco_obj, anns):
for id, ann in enumerate(anns): for id, ann in enumerate(anns):
ann['id'] = id + 1 ann['id'] = id + 1
elif 'bbox' in anns[0] and not anns[0]['bbox'] == []: elif 'bbox' in anns[0] and not anns[0]['bbox'] == []:
res.dataset['categories'] = copy.deepcopy( res.dataset['categories'] = copy.deepcopy(coco_obj.dataset[
coco_obj.dataset['categories']) 'categories'])
for id, ann in enumerate(anns): for id, ann in enumerate(anns):
bb = ann['bbox'] bb = ann['bbox']
x1, x2, y1, y2 = [bb[0], bb[0] + bb[2], bb[1], bb[1] + bb[3]] x1, x2, y1, y2 = [bb[0], bb[0] + bb[2], bb[1], bb[1] + bb[3]]
...@@ -169,8 +169,8 @@ def loadRes(coco_obj, anns): ...@@ -169,8 +169,8 @@ def loadRes(coco_obj, anns):
ann['id'] = id + 1 ann['id'] = id + 1
ann['iscrowd'] = 0 ann['iscrowd'] = 0
elif 'segmentation' in anns[0]: elif 'segmentation' in anns[0]:
res.dataset['categories'] = copy.deepcopy( res.dataset['categories'] = copy.deepcopy(coco_obj.dataset[
coco_obj.dataset['categories']) 'categories'])
for id, ann in enumerate(anns): for id, ann in enumerate(anns):
# now only support compressed RLE format as segmentation results # now only support compressed RLE format as segmentation results
ann['area'] = maskUtils.area(ann['segmentation']) ann['area'] = maskUtils.area(ann['segmentation'])
...@@ -179,8 +179,8 @@ def loadRes(coco_obj, anns): ...@@ -179,8 +179,8 @@ def loadRes(coco_obj, anns):
ann['id'] = id + 1 ann['id'] = id + 1
ann['iscrowd'] = 0 ann['iscrowd'] = 0
elif 'keypoints' in anns[0]: elif 'keypoints' in anns[0]:
res.dataset['categories'] = copy.deepcopy( res.dataset['categories'] = copy.deepcopy(coco_obj.dataset[
coco_obj.dataset['categories']) 'categories'])
for id, ann in enumerate(anns): for id, ann in enumerate(anns):
s = ann['keypoints'] s = ann['keypoints']
x = s[0::3] x = s[0::3]
...@@ -375,8 +375,8 @@ def mask2out(results, clsid2catid, resolution, thresh_binarize=0.5): ...@@ -375,8 +375,8 @@ def mask2out(results, clsid2catid, resolution, thresh_binarize=0.5):
expand_bbox = expand_boxes(bbox, scale) expand_bbox = expand_boxes(bbox, scale)
expand_bbox = expand_bbox.astype(np.int32) expand_bbox = expand_bbox.astype(np.int32)
padded_mask = np.zeros((resolution + 2, resolution + 2), padded_mask = np.zeros(
dtype=np.float32) (resolution + 2, resolution + 2), dtype=np.float32)
for j in range(num): for j in range(num):
xmin, ymin, xmax, ymax = expand_bbox[j].tolist() xmin, ymin, xmax, ymax = expand_bbox[j].tolist()
...@@ -404,7 +404,8 @@ def mask2out(results, clsid2catid, resolution, thresh_binarize=0.5): ...@@ -404,7 +404,8 @@ def mask2out(results, clsid2catid, resolution, thresh_binarize=0.5):
im_mask[y0:y1, x0:x1] = resized_mask[(y0 - ymin):(y1 - ymin), ( im_mask[y0:y1, x0:x1] = resized_mask[(y0 - ymin):(y1 - ymin), (
x0 - xmin):(x1 - xmin)] x0 - xmin):(x1 - xmin)]
segm = mask_util.encode( segm = mask_util.encode(
np.array(im_mask[:, :, np.newaxis], order='F'))[0] np.array(
im_mask[:, :, np.newaxis], order='F'))[0]
catid = clsid2catid[clsid] catid = clsid2catid[clsid]
segm['counts'] = segm['counts'].decode('utf8') segm['counts'] = segm['counts'].decode('utf8')
coco_res = { coco_res = {
...@@ -571,8 +572,8 @@ def prune_zero_padding(gt_box, gt_label, difficult=None): ...@@ -571,8 +572,8 @@ def prune_zero_padding(gt_box, gt_label, difficult=None):
gt_box[i, 2] == 0 and gt_box[i, 3] == 0: gt_box[i, 2] == 0 and gt_box[i, 3] == 0:
break break
valid_cnt += 1 valid_cnt += 1
return (gt_box[:valid_cnt], gt_label[:valid_cnt], return (gt_box[:valid_cnt], gt_label[:valid_cnt], difficult[:valid_cnt]
difficult[:valid_cnt] if difficult is not None else None) if difficult is not None else None)
def bbox_area(bbox, is_bbox_normalized): def bbox_area(bbox, is_bbox_normalized):
...@@ -694,8 +695,9 @@ class DetectionMAP(object): ...@@ -694,8 +695,9 @@ class DetectionMAP(object):
""" """
mAP = 0. mAP = 0.
valid_cnt = 0 valid_cnt = 0
for id, (score_pos, count) in enumerate( for id, (
zip(self.class_score_poss, self.class_gt_counts)): score_pos, count
) in enumerate(zip(self.class_score_poss, self.class_gt_counts)):
if count == 0: continue if count == 0: continue
if len(score_pos) == 0: if len(score_pos) == 0:
valid_cnt += 1 valid_cnt += 1
......
...@@ -116,10 +116,14 @@ coco_pretrain = { ...@@ -116,10 +116,14 @@ coco_pretrain = {
'DeepLabv3p_MobileNetV2_x1.0_COCO': 'DeepLabv3p_MobileNetV2_x1.0_COCO':
'https://bj.bcebos.com/v1/paddleseg/deeplab_mobilenet_x1_0_coco.tgz', 'https://bj.bcebos.com/v1/paddleseg/deeplab_mobilenet_x1_0_coco.tgz',
'DeepLabv3p_Xception65_COCO': 'DeepLabv3p_Xception65_COCO':
'https://paddleseg.bj.bcebos.com/models/xception65_coco.tgz' 'https://paddleseg.bj.bcebos.com/models/xception65_coco.tgz',
'PPYOLO_ResNet50_vd_ssld_COCO':
'https://paddlemodels.bj.bcebos.com/object_detection/ppyolo_2x.pdparams'
} }
cityscapes_pretrain = { cityscapes_pretrain = {
'DeepLabv3p_MobileNetV3_large_x1_0_ssld_CITYSCAPES':
'https://paddleseg.bj.bcebos.com/models/deeplabv3p_mobilenetv3_large_cityscapes.tar.gz',
'DeepLabv3p_MobileNetV2_x1.0_CITYSCAPES': 'DeepLabv3p_MobileNetV2_x1.0_CITYSCAPES':
'https://paddleseg.bj.bcebos.com/models/mobilenet_cityscapes.tgz', 'https://paddleseg.bj.bcebos.com/models/mobilenet_cityscapes.tgz',
'DeepLabv3p_Xception65_CITYSCAPES': 'DeepLabv3p_Xception65_CITYSCAPES':
...@@ -142,7 +146,8 @@ def get_pretrain_weights(flag, class_name, backbone, save_dir): ...@@ -142,7 +146,8 @@ def get_pretrain_weights(flag, class_name, backbone, save_dir):
if flag == 'COCO': if flag == 'COCO':
if class_name == 'DeepLabv3p' and backbone in [ if class_name == 'DeepLabv3p' and backbone in [
'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5', 'Xception41', 'MobileNetV2_x0.25', 'MobileNetV2_x0.5',
'MobileNetV2_x1.5', 'MobileNetV2_x2.0' 'MobileNetV2_x1.5', 'MobileNetV2_x2.0',
'MobileNetV3_large_x1_0_ssld'
]: ]:
model_name = '{}_{}'.format(class_name, backbone) model_name = '{}_{}'.format(class_name, backbone)
logging.warning(warning_info.format(model_name, flag, 'IMAGENET')) logging.warning(warning_info.format(model_name, flag, 'IMAGENET'))
...@@ -226,7 +231,9 @@ def get_pretrain_weights(flag, class_name, backbone, save_dir): ...@@ -226,7 +231,9 @@ def get_pretrain_weights(flag, class_name, backbone, save_dir):
new_save_dir = save_dir new_save_dir = save_dir
if hasattr(paddlex, 'pretrain_dir'): if hasattr(paddlex, 'pretrain_dir'):
new_save_dir = paddlex.pretrain_dir new_save_dir = paddlex.pretrain_dir
if class_name in ['YOLOv3', 'FasterRCNN', 'MaskRCNN', 'DeepLabv3p']: if class_name in [
'YOLOv3', 'FasterRCNN', 'MaskRCNN', 'DeepLabv3p', 'PPYOLO'
]:
backbone = '{}_{}'.format(class_name, backbone) backbone = '{}_{}'.format(class_name, backbone)
backbone = "{}_{}".format(backbone, flag) backbone = "{}_{}".format(backbone, flag)
if flag == 'COCO': if flag == 'COCO':
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -24,8 +24,8 @@ class ConfusionMatrix(object): ...@@ -24,8 +24,8 @@ class ConfusionMatrix(object):
""" """
def __init__(self, num_classes=2, streaming=False): def __init__(self, num_classes=2, streaming=False):
self.confusion_matrix = np.zeros([num_classes, num_classes], self.confusion_matrix = np.zeros(
dtype='int64') [num_classes, num_classes], dtype='int64')
self.num_classes = num_classes self.num_classes = num_classes
self.streaming = streaming self.streaming = streaming
...@@ -42,15 +42,15 @@ class ConfusionMatrix(object): ...@@ -42,15 +42,15 @@ class ConfusionMatrix(object):
pred = np.asarray(pred)[mask] pred = np.asarray(pred)[mask]
one = np.ones_like(pred) one = np.ones_like(pred)
# Accumuate ([row=label, col=pred], 1) into sparse matrix # Accumuate ([row=label, col=pred], 1) into sparse matrix
spm = csr_matrix((one, (label, pred)), spm = csr_matrix(
shape=(self.num_classes, self.num_classes)) (one, (label, pred)), shape=(self.num_classes, self.num_classes))
spm = spm.todense() spm = spm.todense()
self.confusion_matrix += spm self.confusion_matrix += spm
def zero_matrix(self): def zero_matrix(self):
""" Clear confusion matrix """ """ Clear confusion matrix """
self.confusion_matrix = np.zeros([self.num_classes, self.num_classes], self.confusion_matrix = np.zeros(
dtype='int64') [self.num_classes, self.num_classes], dtype='int64')
def mean_iou(self): def mean_iou(self):
iou_list = [] iou_list = []
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -404,8 +404,9 @@ def draw_pr_curve(eval_details_file=None, ...@@ -404,8 +404,9 @@ def draw_pr_curve(eval_details_file=None,
plt.plot(x, sr_array, color=color, label=nm, linewidth=1) plt.plot(x, sr_array, color=color, label=nm, linewidth=1)
plt.legend(loc="lower left", fontsize=5) plt.legend(loc="lower left", fontsize=5)
plt.savefig( plt.savefig(
os.path.join(save_dir, os.path.join(
"./{}_pr_curve(iou-{}).png".format(style, iou_thresh)), save_dir,
"./{}_pr_curve(iou-{}).png".format(style, iou_thresh)),
dpi=800) dpi=800)
plt.close() plt.close()
......
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -15,21 +15,11 @@ ...@@ -15,21 +15,11 @@
from __future__ import absolute_import from __future__ import absolute_import
import math import math
import tqdm import tqdm
import os.path as osp
import numpy as np
from multiprocessing.pool import ThreadPool
import paddle.fluid as fluid
import paddlex.utils.logging as logging
import paddlex import paddlex
import copy from .ppyolo import PPYOLO
from paddlex.cv.transforms import arrange_transforms
from paddlex.cv.datasets import generate_minibatch
from .base import BaseAPI
from collections import OrderedDict
from .utils.detection_eval import eval_results, bbox2out
class YOLOv3(BaseAPI): class YOLOv3(PPYOLO):
"""构建YOLOv3,并实现其训练、评估、预测和模型导出。 """构建YOLOv3,并实现其训练、评估、预测和模型导出。
Args: Args:
...@@ -45,7 +35,7 @@ class YOLOv3(BaseAPI): ...@@ -45,7 +35,7 @@ class YOLOv3(BaseAPI):
nms_score_threshold (float): 检测框的置信度得分阈值,置信度得分低于阈值的框应该被忽略。默认为0.01。 nms_score_threshold (float): 检测框的置信度得分阈值,置信度得分低于阈值的框应该被忽略。默认为0.01。
nms_topk (int): 进行NMS时,根据置信度保留的最大检测框数。默认为1000。 nms_topk (int): 进行NMS时,根据置信度保留的最大检测框数。默认为1000。
nms_keep_topk (int): 进行NMS后,每个图像要保留的总检测框数。默认为100。 nms_keep_topk (int): 进行NMS后,每个图像要保留的总检测框数。默认为100。
nms_iou_threshold (float): 进行NMS时,用于剔除检测框IOU的阈值。默认为0.45。 nms_iou_threshold (float): 进行NMS时,用于剔除检测框IoU的阈值。默认为0.45。
label_smooth (bool): 是否使用label smooth。默认值为False。 label_smooth (bool): 是否使用label smooth。默认值为False。
train_random_shapes (list|tuple): 训练时从列表中随机选择图像大小。默认值为[320, 352, 384, 416, 448, 480, 512, 544, 576, 608]。 train_random_shapes (list|tuple): 训练时从列表中随机选择图像大小。默认值为[320, 352, 384, 416, 448, 480, 512, 544, 576, 608]。
""" """
...@@ -65,12 +55,12 @@ class YOLOv3(BaseAPI): ...@@ -65,12 +55,12 @@ class YOLOv3(BaseAPI):
320, 352, 384, 416, 448, 480, 512, 544, 576, 608 320, 352, 384, 416, 448, 480, 512, 544, 576, 608
]): ]):
self.init_params = locals() self.init_params = locals()
super(YOLOv3, self).__init__('detector')
backbones = [ backbones = [
'DarkNet53', 'ResNet34', 'MobileNetV1', 'MobileNetV3_large' 'DarkNet53', 'ResNet34', 'MobileNetV1', 'MobileNetV3_large'
] ]
assert backbone in backbones, "backbone should be one of {}".format( assert backbone in backbones, "backbone should be one of {}".format(
backbones) backbones)
super(PPYOLO, self).__init__('detector')
self.backbone = backbone self.backbone = backbone
self.num_classes = num_classes self.num_classes = num_classes
self.anchors = anchors self.anchors = anchors
...@@ -84,6 +74,16 @@ class YOLOv3(BaseAPI): ...@@ -84,6 +74,16 @@ class YOLOv3(BaseAPI):
self.sync_bn = True self.sync_bn = True
self.train_random_shapes = train_random_shapes self.train_random_shapes = train_random_shapes
self.fixed_input_shape = None self.fixed_input_shape = None
self.use_fine_grained_loss = False
self.use_coord_conv = False
self.use_iou_aware = False
self.use_spp = False
self.use_drop_block = False
self.use_iou_loss = False
self.scale_x_y = 1.
self.use_matrix_nms = False
self.use_ema = False
self.with_dcn_v2 = False
def _get_backbone(self, backbone_name): def _get_backbone(self, backbone_name):
if backbone_name == 'DarkNet53': if backbone_name == 'DarkNet53':
...@@ -104,59 +104,6 @@ class YOLOv3(BaseAPI): ...@@ -104,59 +104,6 @@ class YOLOv3(BaseAPI):
norm_type='sync_bn', model_name=model_name) norm_type='sync_bn', model_name=model_name)
return backbone return backbone
def build_net(self, mode='train'):
model = paddlex.cv.nets.detection.YOLOv3(
backbone=self._get_backbone(self.backbone),
num_classes=self.num_classes,
mode=mode,
anchors=self.anchors,
anchor_masks=self.anchor_masks,
ignore_threshold=self.ignore_threshold,
label_smooth=self.label_smooth,
nms_score_threshold=self.nms_score_threshold,
nms_topk=self.nms_topk,
nms_keep_topk=self.nms_keep_topk,
nms_iou_threshold=self.nms_iou_threshold,
train_random_shapes=self.train_random_shapes,
fixed_input_shape=self.fixed_input_shape)
inputs = model.generate_inputs()
model_out = model.build_net(inputs)
outputs = OrderedDict([('bbox', model_out)])
if mode == 'train':
self.optimizer.minimize(model_out)
outputs = OrderedDict([('loss', model_out)])
return inputs, outputs
def default_optimizer(self, learning_rate, warmup_steps, warmup_start_lr,
lr_decay_epochs, lr_decay_gamma,
num_steps_each_epoch):
if warmup_steps > lr_decay_epochs[0] * num_steps_each_epoch:
logging.error(
"In function train(), parameters should satisfy: warmup_steps <= lr_decay_epochs[0]*num_samples_in_train_dataset",
exit=False)
logging.error(
"See this doc for more information: https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/appendix/parameters.md#notice",
exit=False)
logging.error(
"warmup_steps should less than {} or lr_decay_epochs[0] greater than {}, please modify 'lr_decay_epochs' or 'warmup_steps' in train function".
format(lr_decay_epochs[0] * num_steps_each_epoch, warmup_steps
// num_steps_each_epoch))
boundaries = [b * num_steps_each_epoch for b in lr_decay_epochs]
values = [(lr_decay_gamma**i) * learning_rate
for i in range(len(lr_decay_epochs) + 1)]
lr_decay = fluid.layers.piecewise_decay(
boundaries=boundaries, values=values)
lr_warmup = fluid.layers.linear_lr_warmup(
learning_rate=lr_decay,
warmup_steps=warmup_steps,
start_lr=warmup_start_lr,
end_lr=learning_rate)
optimizer = fluid.optimizer.Momentum(
learning_rate=lr_warmup,
momentum=0.9,
regularization=fluid.regularizer.L2DecayRegularizer(5e-04))
return optimizer
def train(self, def train(self,
num_epochs, num_epochs,
train_dataset, train_dataset,
...@@ -214,259 +161,11 @@ class YOLOv3(BaseAPI): ...@@ -214,259 +161,11 @@ class YOLOv3(BaseAPI):
ValueError: 评估类型不在指定列表中。 ValueError: 评估类型不在指定列表中。
ValueError: 模型从inference model进行加载。 ValueError: 模型从inference model进行加载。
""" """
if not self.trainable:
raise ValueError("Model is not trainable from load_model method.")
if metric is None:
if isinstance(train_dataset, paddlex.datasets.CocoDetection):
metric = 'COCO'
elif isinstance(train_dataset, paddlex.datasets.VOCDetection) or \
isinstance(train_dataset, paddlex.datasets.EasyDataDet):
metric = 'VOC'
else:
raise ValueError(
"train_dataset should be datasets.VOCDetection or datasets.COCODetection or datasets.EasyDataDet."
)
assert metric in ['COCO', 'VOC'], "Metric only support 'VOC' or 'COCO'"
self.metric = metric
self.labels = train_dataset.labels
# 构建训练网络
if optimizer is None:
# 构建默认的优化策略
num_steps_each_epoch = train_dataset.num_samples // train_batch_size
optimizer = self.default_optimizer(
learning_rate=learning_rate,
warmup_steps=warmup_steps,
warmup_start_lr=warmup_start_lr,
lr_decay_epochs=lr_decay_epochs,
lr_decay_gamma=lr_decay_gamma,
num_steps_each_epoch=num_steps_each_epoch)
self.optimizer = optimizer
# 构建训练、验证、预测网络
self.build_program()
# 初始化网络权重
self.net_initialize(
startup_prog=fluid.default_startup_program(),
pretrain_weights=pretrain_weights,
save_dir=save_dir,
sensitivities_file=sensitivities_file,
eval_metric_loss=eval_metric_loss,
resume_checkpoint=resume_checkpoint)
# 训练
self.train_loop(
num_epochs=num_epochs,
train_dataset=train_dataset,
train_batch_size=train_batch_size,
eval_dataset=eval_dataset,
save_interval_epochs=save_interval_epochs,
log_interval_steps=log_interval_steps,
save_dir=save_dir,
use_vdl=use_vdl,
early_stop=early_stop,
early_stop_patience=early_stop_patience)
def evaluate(self,
eval_dataset,
batch_size=1,
epoch_id=None,
metric=None,
return_details=False):
"""评估。
Args:
eval_dataset (paddlex.datasets): 验证数据读取器。
batch_size (int): 验证数据批大小。默认为1。
epoch_id (int): 当前评估模型所在的训练轮数。
metric (bool): 训练过程中评估的方式,取值范围为['COCO', 'VOC']。默认为None,
根据用户传入的Dataset自动选择,如为VOCDetection,则metric为'VOC';
如为COCODetection,则metric为'COCO'。
return_details (bool): 是否返回详细信息。
Returns:
tuple (metrics, eval_details) | dict (metrics): 当return_details为True时,返回(metrics, eval_details),
当return_details为False时,返回metrics。metrics为dict,包含关键字:'bbox_mmap'或者’bbox_map‘,
分别表示平均准确率平均值在各个IoU阈值下的结果取平均值的结果(mmAP)、平均准确率平均值(mAP)。
eval_details为dict,包含关键字:'bbox',对应元素预测结果列表,每个预测结果由图像id、
预测框类别id、预测框坐标、预测框得分;’gt‘:真实标注框相关信息。
"""
arrange_transforms(
model_type=self.model_type,
class_name=self.__class__.__name__,
transforms=eval_dataset.transforms,
mode='eval')
if metric is None:
if hasattr(self, 'metric') and self.metric is not None:
metric = self.metric
else:
if isinstance(eval_dataset, paddlex.datasets.CocoDetection):
metric = 'COCO'
elif isinstance(eval_dataset, paddlex.datasets.VOCDetection):
metric = 'VOC'
else:
raise Exception(
"eval_dataset should be datasets.VOCDetection or datasets.COCODetection."
)
assert metric in ['COCO', 'VOC'], "Metric only support 'VOC' or 'COCO'"
total_steps = math.ceil(eval_dataset.num_samples * 1.0 / batch_size)
results = list()
data_generator = eval_dataset.generator(
batch_size=batch_size, drop_last=False)
logging.info(
"Start to evaluating(total_samples={}, total_steps={})...".format(
eval_dataset.num_samples, total_steps))
for step, data in tqdm.tqdm(
enumerate(data_generator()), total=total_steps):
images = np.array([d[0] for d in data])
im_sizes = np.array([d[1] for d in data])
feed_data = {'image': images, 'im_size': im_sizes}
with fluid.scope_guard(self.scope):
outputs = self.exe.run(
self.test_prog,
feed=[feed_data],
fetch_list=list(self.test_outputs.values()),
return_numpy=False)
res = {
'bbox': (np.array(outputs[0]),
outputs[0].recursive_sequence_lengths())
}
res_id = [np.array([d[2]]) for d in data]
res['im_id'] = (res_id, [])
if metric == 'VOC':
res_gt_box = [d[3].reshape(-1, 4) for d in data]
res_gt_label = [d[4].reshape(-1, 1) for d in data]
res_is_difficult = [d[5].reshape(-1, 1) for d in data]
res_id = [np.array([d[2]]) for d in data]
res['gt_box'] = (res_gt_box, [])
res['gt_label'] = (res_gt_label, [])
res['is_difficult'] = (res_is_difficult, [])
results.append(res)
logging.debug("[EVAL] Epoch={}, Step={}/{}".format(epoch_id, step +
1, total_steps))
box_ap_stats, eval_details = eval_results(
results, metric, eval_dataset.coco_gt, with_background=False)
evaluate_metrics = OrderedDict(
zip(['bbox_mmap'
if metric == 'COCO' else 'bbox_map'], box_ap_stats))
if return_details:
return evaluate_metrics, eval_details
return evaluate_metrics
@staticmethod
def _preprocess(images, transforms, model_type, class_name, thread_num=1):
arrange_transforms(
model_type=model_type,
class_name=class_name,
transforms=transforms,
mode='test')
pool = ThreadPool(thread_num)
batch_data = pool.map(transforms, images)
pool.close()
pool.join()
padding_batch = generate_minibatch(batch_data)
im = np.array(
[data[0] for data in padding_batch],
dtype=padding_batch[0][0].dtype)
im_size = np.array([data[1] for data in padding_batch], dtype=np.int32)
return im, im_size
@staticmethod
def _postprocess(res, batch_size, num_classes, labels):
clsid2catid = dict({i: i for i in range(num_classes)})
xywh_results = bbox2out([res], clsid2catid)
preds = [[] for i in range(batch_size)]
for xywh_res in xywh_results:
image_id = xywh_res['image_id']
del xywh_res['image_id']
xywh_res['category'] = labels[xywh_res['category_id']]
preds[image_id].append(xywh_res)
return preds
def predict(self, img_file, transforms=None):
"""预测。
Args:
img_file (str|np.ndarray): 预测图像路径,或者是解码后的排列格式为(H, W, C)且类型为float32且为BGR格式的数组。
transforms (paddlex.det.transforms): 数据预处理操作。
Returns:
list: 预测结果列表,每个预测结果由预测框类别标签、
预测框类别名称、预测框坐标(坐标格式为[xmin, ymin, w, h])、
预测框得分组成。
"""
if transforms is None and not hasattr(self, 'test_transforms'):
raise Exception("transforms need to be defined, now is None.")
if isinstance(img_file, (str, np.ndarray)):
images = [img_file]
else:
raise Exception("img_file must be str/np.ndarray")
if transforms is None:
transforms = self.test_transforms
im, im_size = YOLOv3._preprocess(images, transforms, self.model_type,
self.__class__.__name__)
with fluid.scope_guard(self.scope):
result = self.exe.run(self.test_prog,
feed={'image': im,
'im_size': im_size},
fetch_list=list(self.test_outputs.values()),
return_numpy=False,
use_program_cache=True)
res = {
k: (np.array(v), v.recursive_sequence_lengths())
for k, v in zip(list(self.test_outputs.keys()), result)
}
res['im_id'] = (np.array(
[[i] for i in range(len(images))]).astype('int32'), [[]])
preds = YOLOv3._postprocess(res,
len(images), self.num_classes, self.labels)
return preds[0]
def batch_predict(self, img_file_list, transforms=None, thread_num=2):
"""预测。
Args:
img_file_list (list|tuple): 对列表(或元组)中的图像同时进行预测,列表中的元素可以是图像路径,也可以是解码后的排列格式为(H,W,C)
且类型为float32且为BGR格式的数组。
transforms (paddlex.det.transforms): 数据预处理操作。
thread_num (int): 并发执行各图像预处理时的线程数。
Returns:
list: 每个元素都为列表,表示各图像的预测结果。在各图像的预测结果列表中,每个预测结果由预测框类别标签、
预测框类别名称、预测框坐标(坐标格式为[xmin, ymin, w, h])、
预测框得分组成。
"""
if transforms is None and not hasattr(self, 'test_transforms'):
raise Exception("transforms need to be defined, now is None.")
if not isinstance(img_file_list, (list, tuple)):
raise Exception("im_file must be list/tuple")
if transforms is None:
transforms = self.test_transforms
im, im_size = YOLOv3._preprocess(img_file_list, transforms,
self.model_type,
self.__class__.__name__, thread_num)
with fluid.scope_guard(self.scope):
result = self.exe.run(self.test_prog,
feed={'image': im,
'im_size': im_size},
fetch_list=list(self.test_outputs.values()),
return_numpy=False,
use_program_cache=True)
res = { return super(YOLOv3, self).train(
k: (np.array(v), v.recursive_sequence_lengths()) num_epochs, train_dataset, train_batch_size, eval_dataset,
for k, v in zip(list(self.test_outputs.keys()), result) save_interval_epochs, log_interval_steps, save_dir,
} pretrain_weights, optimizer, learning_rate, warmup_steps,
res['im_id'] = (np.array( warmup_start_lr, lr_decay_epochs, lr_decay_gamma, metric, use_vdl,
[[i] for i in range(len(img_file_list))]).astype('int32'), [[]]) sensitivities_file, eval_metric_loss, early_stop,
preds = YOLOv3._postprocess(res, early_stop_patience, resume_checkpoint, False)
len(img_file_list), self.num_classes,
self.labels)
return preds
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. # copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
......
#copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. #copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
# #
#Licensed under the Apache License, Version 2.0 (the "License"); #Licensed under the Apache License, Version 2.0 (the "License");
#you may not use this file except in compliance with the License. #you may not use this file except in compliance with the License.
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册