diff --git a/README.md b/README.md index 644769e6dce6a7133e1532379ce36a8a78f569f4..c0f87822be00e6ed5ab013f98bc46c262b55bfaf 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@

- PaddleX + PaddleX

[![License](https://img.shields.io/badge/license-Apache%202-red.svg)](LICENSE) @@ -8,66 +8,77 @@ ![support os](https://img.shields.io/badge/os-linux%2C%20win%2C%20mac-yellow.svg) ![QQGroup](https://img.shields.io/badge/QQ_Group-1045148026-52B6EF?style=social&logo=tencent-qq&logoColor=000&logoWidth=20) -PaddleX是基于飞桨核心框架、开发套件和工具组件的深度学习全流程开发工具。具备**全流程打通**、**融合产业实践**、**易用易集成**三大特点。 +**PaddleX--飞桨全功能开发套件**，集成了飞桨视觉套件（PaddleClas、PaddleDetection、PaddleSeg）、模型压缩工具PaddleSlim、可视化分析工具VisualDL、轻量化推理引擎Paddle Lite 等核心模块的能力，同时融合飞桨团队丰富的实际经验及技术积累，将深度学习开发全流程，从数据准备、模型训练与优化到多端部署实现了端到端打通，为开发者提供飞桨全流程开发的最佳实践。 -## 特点 +**PaddleX 提供了最简化的API设计，并官方实现GUI供大家下载使用**，最大程度降低开发者使用门槛。开发者既可以应用**PaddleX GUI**快速体验深度学习模型开发的全流程，也可以直接使用 **PaddleX API** 更灵活地进行开发。 -- **全流程打通** - - **数据准备**：支持[EasyData智能数据服务平台](https://ai.baidu.com/easydata/)数据协议，通过平台便捷完成智能标注,低质数据清洗工作, 同时兼容主流标注工具协议, 助力开发者更快完成数据准备工作。 - - **模型训练**：集成[PaddleClas](https://github.com/PaddlePaddle/PaddleClas), [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), [PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)视觉开发套件，丰富的高质量预训练模型，更快实现工业级模型效果。 - - **模型调优**：内置模型可解释性模块、[VisualDL](https://github.com/PaddlePaddle/VisualDL)可视化分析组件, 提供丰富的信息更好地理解模型，优化模型。 - - **多端安全部署**：内置[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)模型压缩工具和**模型加密部署模块**，结合Paddle Inference或[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite)便捷完成高性能的多端安全部署。 +更进一步的，如果用户需要根据自己场景及需求，定制化地对PaddleX 进行改造或集成，PaddleX 也提供很好的支持。 -- **融合产业实践** - - 精选飞桨产业实践的成熟模型结构，开放案例实践教程，加速开发者产业落地。 +## PaddleX 三大特点 -- **易用易集成** - - 统一易用的全流程API，5步完成模型训练，10行代码实现Python/C++高性能部署。 - - 提供以PaddleX为核心集成的跨平台可视化工具PaddleX-GUI，快速体验飞桨深度学习全流程。 +### 全流程打通 +- **数据准备**：兼容ImageNet、VOC、COCO等常用数据协议, 同时与Labelme、精灵标注助手、[EasyData智能数据服务平台](https://ai.baidu.com/easydata/)等无缝衔接，全方位助力开发者更快完成数据准备工作。 -## 安装 +- **数据预处理及增强**：提供极简的图像预处理和增强方法--Transforms，适配imgaug图像增强库，支持上百种数据增强策略，是开发者快速缓解小样本数据训练的问题。 + +- **模型训练**：集成[PaddleClas](https://github.com/PaddlePaddle/PaddleClas), [PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection), [PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)视觉开发套件，提供大量精选的、经过产业实践的高质量预训练模型，使开发者更快实现工业级模型效果。 + +- **模型调优**：内置模型可解释性模块、[VisualDL](https://github.com/PaddlePaddle/VisualDL)可视化分析工具。使开发者可以更直观的理解模型的特征提取区域、训练过程参数变化，从而快速优化模型。 -PaddleX提供两种开发模式，满足不同场景和用户需求： +- **多端安全部署**：内置[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)模型压缩工具和**模型加密部署模块**，与飞桨原生预测库Paddle Inference及高性能端侧推理引擎[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) 无缝打通，使开发者快速实现模型的多端、高性能、安全部署。 -- **Python开发模式：** 通过Python API方式完成全流程使用或集成，该模型提供全面、灵活、开放的深度学习功能，有更高的定制化空间。 +### 融合产业实践 -- **Padlde-GUI模式：** PaddleX-GUI支持`Python开发模式`下的常用功能，以更低门槛的方式快速完成产业验证的模型训练。 +- **产业验证**：经过**质检**、**安防**、**巡检**、**遥感**、**零售**、**医疗**等十多个行业实际应用场景验证，适配行业数据格式及部署环境要求。 +- **经验沉淀**：沉淀产业实践实际经验，**提供丰富的案例实践教程**，加速开发者产业落地。 +- **产业开发者共建**：吸收实际产业开发者贡献代码，源于产业，回馈产业。 -开发者可根据自身需要按需选择不同的模式进行安装使用。 -### Python开发模式安装 +## 易用易集成 + +- **易用**：统一的全流程API，5步即可完成模型训练，10行代码实现Python/C++高性能部署。 +- **易集成**：支持开发者自主改造、集成，开发出适用于自己产业需求的产品。并官方提供基于 PaddleX API 开发的跨平台可视化工具-- **PaddleX GUI**，使开发者快速体验飞桨深度学习开发全流程，并启发用户进行定制化开发。 + +## 安装 -**前置依赖** -* paddlepaddle >= 1.8.0 -* python >= 3.5 -* cython -* pycocotools +**PaddleX提供两种开发模式，满足用户的不同需求：** + +1. **Python开发模式：** 通过简洁易懂的Python API，在兼顾功能全面性、开发灵活性、集成方便性的基础上，给开发者最流畅的深度学习开发体验。
+ + **前置依赖** +> - paddlepaddle >= 1.8.0 +> - python >= 3.5 +> - cython +> - pycocotools ``` pip install paddlex -i https://mirror.baidu.com/pypi/simple ``` -安装的相关问题参考[PaddleX安装](https://paddlex.readthedocs.io/zh_CN/latest/install.html) +详细安装方法请参考[PaddleX安装](https://paddlex.readthedocs.io/zh_CN/develop/install.html) + + +2. **Padlde GUI模式：** 无代码开发的可视化客户端，应用Paddle API实现，使开发者快速进行产业项目验证，并为用户开发自有深度学习软件/应用提供参照。 + +- 前往[PaddleX官网](https://www.paddlepaddle.org.cn/paddle/paddlex)，申请下载Paddle X GUI一键绿色安装包。 -### PaddleX-GUI安装 +- 前往[PaddleX GUI使用教程](./docs/gui/how_to_use.md)了解PaddleX GUI使用详情。 -进入PaddleX官网[下载使用](https://www.paddlepaddle.org.cn/paddle/paddlex)，申请下载绿色安装包，开箱即用。 -Paddle-GUI的使用教程可参考[PaddleX-GUI模式使用教程](https://paddlex.readthedocs.io/zh_CN/latest/paddlex_gui/index.html) -## 使用文档 +## 完整使用文档及API说明 -推荐访问[PaddleX在线使用文档](https://paddlex.readthedocs.io/zh_CN/latest/index.html)，快速查阅读使用教程和API文档说明。 +- [完整PaddleX在线使用文档目录](https://paddlex.readthedocs.io/zh_CN/develop/index.html) -- [10分钟快速上手使用](https://paddlex.readthedocs.io/zh_CN/latest/quick_start.html) -- [PaddleX全流程开发教程](https://paddlex.readthedocs.io/zh_CN/latest/tutorials/index.html) -- [PaddleX视觉方案介绍](https://paddlex.readthedocs.io/zh_CN/latest/cv_solutions.html) -- [PaddleX API参考文档](https://paddlex.readthedocs.io/zh_CN/latest/apis/index.html) +- [10分钟快速上手系列教程](https://paddlex.readthedocs.io/zh_CN/develop/quick_start.html) +- [PaddleX模型训练教程集合](https://paddlex.readthedocs.io/zh_CN/develop/train/index.html) +- [PaddleX API接口说明](https://paddlex.readthedocs.io/zh_CN/develop/apis/index.html) -## 在线教程 +## 在线项目示例 -基于AIStudio平台，快速在线体验PaddleX的Python开发模式教程。 +为了使开发者更快掌握PaddleX API，我们创建了一系列完整的示例教程，您可通过AIStudio一站式开发平台，快速在线运行PaddleX的项目。 +- [PaddleX快速上手CV模型训练](https://aistudio.baidu.com/aistudio/projectdetail/450925) - [PaddleX快速上手——MobileNetV3-ssld 化妆品分类](https://aistudio.baidu.com/aistudio/projectdetail/450220) - [PaddleX快速上手——Faster-RCNN AI识虫](https://aistudio.baidu.com/aistudio/projectdetail/439888) - [PaddleX快速上手——DeepLabv3+ 视盘分割](https://aistudio.baidu.com/aistudio/projectdetail/440197) @@ -76,15 +87,16 @@ Paddle-GUI的使用教程可参考[PaddleX-GUI模式使用教程](https://paddle - 项目官网: https://www.paddlepaddle.org.cn/paddle/paddlex - PaddleX用户交流群: 1045148026 (手机QQ扫描如下二维码快速加入) -QQGroup +QQGroup -## FAQ +## [FAQ](./docs/gui/faq.md) ## 更新日志 -* 2020.05.20 +> [历史版本及更新内容](https://paddlex.readthedocs.io/zh_CN/develop/change_log.html) - **`v1.0.0`** - * 正式版本发布。 +- 2020.07.12 v1.0.8 +- 2020.05.20 v1.0.0 +- 2020.05.17 v0.1.8 ## 贡献代码 diff --git a/deploy/README.md b/deploy/README.md index 515b1a16878efe8b1d18622aa811a335a285cdac..7fe3219882c3c8d863824829baf6742b74759d2f 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -1,7 +1,16 @@ -# 多端安全部署 +# 模型部署 本目录为PaddleX模型部署代码，编译和使用教程参考： -- [服务端部署(支持Python部署、C++部署、模型加密部署)](../docs/tutorials/deploy/deploy_server/) -- [OpenVINO部署](../docs/tutorials/deploy/deploy_openvino.md) -- [移动端部署](../docs/tutorials/deploy/deploy_lite.md) +- [服务端部署](../docs/deploy/server/) + - [Python部署](../docs/deploy/server/python.md) + - [C++部署](../docs/deploy/server/cpp/) + - [Windows平台部署](../docs/deploy/server/cpp/windows.md) + - [Linux平台部署](../docs/deploy/server/cpp/linux.md) + - [模型加密部署](../docs/deploy/server/encryption.md) +- [Nvidia Jetson开发板部署](../docs/deploy/nvidia-jetson.md) +- [移动端部署](../docs/deploy/paddlelite/) + - [模型压缩](../docs/deploy/paddlelite/slim) + - [模型量化](../docs/deploy/paddlelite/slim/quant.md) + - [模型裁剪](../docs/deploy/paddlelite/slim/prune.md) + - [Android平台](../docs/deploy/paddlelite/android.md) diff --git a/deploy/cpp/demo/classifier.cpp b/deploy/cpp/demo/classifier.cpp index 6fd354d3f9cb6a366f0efb0b31e7ae073a90b4ad..db3687492789f47a3bb49643b87f9b946f05137d 100644 --- a/deploy/cpp/demo/classifier.cpp +++ b/deploy/cpp/demo/classifier.cpp @@ -62,8 +62,6 @@ int main(int argc, char** argv) { FLAGS_use_ir_optim); // 进行预测 - double total_running_time_s = 0.0; - double total_imread_time_s = 0.0; int imgs = 1; if (FLAGS_image_list != "") { std::ifstream inf(FLAGS_image_list); @@ -79,7 +77,6 @@ int main(int argc, char** argv) { } imgs = image_paths.size(); for (int i = 0; i < image_paths.size(); i += FLAGS_batch_size) { - auto start = system_clock::now(); // 读图像 int im_vec_size = std::min(static_cast(image_paths.size()), i + FLAGS_batch_size); @@ -91,19 +88,7 @@ int main(int argc, char** argv) { for (int j = i; j < im_vec_size; ++j) { im_vec[j - i] = std::move(cv::imread(image_paths[j], 1)); } - auto imread_end = system_clock::now(); model.predict(im_vec, &results, thread_num); - - auto imread_duration = duration_cast(imread_end - start); - total_imread_time_s += static_cast(imread_duration.count()) * - microseconds::period::num / - microseconds::period::den; - - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; for (int j = i; j < im_vec_size; ++j) { std::cout << "Path:" << image_paths[j] << ", predict label: " << results[j - i].category @@ -112,23 +97,12 @@ int main(int argc, char** argv) { } } } else { - auto start = system_clock::now(); PaddleX::ClsResult result; cv::Mat im = cv::imread(FLAGS_image, 1); model.predict(im, &result); - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; std::cout << "Predict label: " << result.category << ", label_id:" << result.category_id << ", score: " << result.score << std::endl; } - std::cout << "Total running time: " << total_running_time_s - << " s, average running time: " << total_running_time_s / imgs - << " s/img, total read img time: " << total_imread_time_s - << " s, average read time: " << total_imread_time_s / imgs - << " s/img, batch_size = " << FLAGS_batch_size << std::endl; return 0; } diff --git a/deploy/cpp/demo/detector.cpp b/deploy/cpp/demo/detector.cpp index 54f93d2995fa24af73bba2855b6b26466129fa20..32fbaafddc9cdbcfddf69164197143238bf26ca4 100644 --- a/deploy/cpp/demo/detector.cpp +++ b/deploy/cpp/demo/detector.cpp @@ -65,11 +65,7 @@ int main(int argc, char** argv) { FLAGS_gpu_id, FLAGS_key, FLAGS_use_ir_optim); - - double total_running_time_s = 0.0; - double total_imread_time_s = 0.0; int imgs = 1; - auto colormap = PaddleX::GenerateColorMap(model.labels.size()); std::string save_dir = "output"; // 进行预测 if (FLAGS_image_list != "") { @@ -85,7 +81,6 @@ int main(int argc, char** argv) { } imgs = image_paths.size(); for (int i = 0; i < image_paths.size(); i += FLAGS_batch_size) { - auto start = system_clock::now(); int im_vec_size = std::min(static_cast(image_paths.size()), i + FLAGS_batch_size); std::vector im_vec(im_vec_size - i); @@ -96,17 +91,7 @@ int main(int argc, char** argv) { for (int j = i; j < im_vec_size; ++j) { im_vec[j - i] = std::move(cv::imread(image_paths[j], 1)); } - auto imread_end = system_clock::now(); model.predict(im_vec, &results, thread_num); - auto imread_duration = duration_cast(imread_end - start); - total_imread_time_s += static_cast(imread_duration.count()) * - microseconds::period::num / - microseconds::period::den; - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; // 输出结果目标框 for (int j = 0; j < im_vec_size - i; ++j) { for (int k = 0; k < results[j].boxes.size(); ++k) { @@ -124,7 +109,7 @@ int main(int argc, char** argv) { // 可视化 for (int j = 0; j < im_vec_size - i; ++j) { cv::Mat vis_img = PaddleX::Visualize( - im_vec[j], results[j], model.labels, colormap, FLAGS_threshold); + im_vec[j], results[j], model.labels, FLAGS_threshold); std::string save_path = PaddleX::generate_save_path(FLAGS_save_dir, image_paths[i + j]); cv::imwrite(save_path, vis_img); @@ -132,15 +117,9 @@ int main(int argc, char** argv) { } } } else { - auto start = system_clock::now(); PaddleX::DetResult result; cv::Mat im = cv::imread(FLAGS_image, 1); model.predict(im, &result); - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; // 输出结果目标框 for (int i = 0; i < result.boxes.size(); ++i) { std::cout << "image file: " << FLAGS_image << std::endl; @@ -155,7 +134,7 @@ int main(int argc, char** argv) { // 可视化 cv::Mat vis_img = - PaddleX::Visualize(im, result, model.labels, colormap, FLAGS_threshold); + PaddleX::Visualize(im, result, model.labels, FLAGS_threshold); std::string save_path = PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_image); cv::imwrite(save_path, vis_img); @@ -163,11 +142,5 @@ int main(int argc, char** argv) { std::cout << "Visualized output saved as " << save_path << std::endl; } - std::cout << "Total running time: " << total_running_time_s - << " s, average running time: " << total_running_time_s / imgs - << " s/img, total read img time: " << total_imread_time_s - << " s, average read img time: " << total_imread_time_s / imgs - << " s, batch_size = " << FLAGS_batch_size << std::endl; - return 0; } diff --git a/deploy/cpp/demo/segmenter.cpp b/deploy/cpp/demo/segmenter.cpp index 90adb5aea860bf5ad9f6cb9019990a188c37f871..b3b8fad9ac2dce33722c71d9d50d354349298230 100644 --- a/deploy/cpp/demo/segmenter.cpp +++ b/deploy/cpp/demo/segmenter.cpp @@ -62,11 +62,7 @@ int main(int argc, char** argv) { FLAGS_gpu_id, FLAGS_key, FLAGS_use_ir_optim); - - double total_running_time_s = 0.0; - double total_imread_time_s = 0.0; int imgs = 1; - auto colormap = PaddleX::GenerateColorMap(model.labels.size()); // 进行预测 if (FLAGS_image_list != "") { std::ifstream inf(FLAGS_image_list); @@ -81,7 +77,6 @@ int main(int argc, char** argv) { } imgs = image_paths.size(); for (int i = 0; i < image_paths.size(); i += FLAGS_batch_size) { - auto start = system_clock::now(); int im_vec_size = std::min(static_cast(image_paths.size()), i + FLAGS_batch_size); std::vector im_vec(im_vec_size - i); @@ -92,21 +87,11 @@ int main(int argc, char** argv) { for (int j = i; j < im_vec_size; ++j) { im_vec[j - i] = std::move(cv::imread(image_paths[j], 1)); } - auto imread_end = system_clock::now(); model.predict(im_vec, &results, thread_num); - auto imread_duration = duration_cast(imread_end - start); - total_imread_time_s += static_cast(imread_duration.count()) * - microseconds::period::num / - microseconds::period::den; - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; // 可视化 for (int j = 0; j < im_vec_size - i; ++j) { cv::Mat vis_img = - PaddleX::Visualize(im_vec[j], results[j], model.labels, colormap); + PaddleX::Visualize(im_vec[j], results[j], model.labels); std::string save_path = PaddleX::generate_save_path(FLAGS_save_dir, image_paths[i + j]); cv::imwrite(save_path, vis_img); @@ -114,28 +99,16 @@ int main(int argc, char** argv) { } } } else { - auto start = system_clock::now(); PaddleX::SegResult result; cv::Mat im = cv::imread(FLAGS_image, 1); model.predict(im, &result); - auto end = system_clock::now(); - auto duration = duration_cast(end - start); - total_running_time_s += static_cast(duration.count()) * - microseconds::period::num / - microseconds::period::den; // 可视化 - cv::Mat vis_img = PaddleX::Visualize(im, result, model.labels, colormap); + cv::Mat vis_img = PaddleX::Visualize(im, result, model.labels); std::string save_path = PaddleX::generate_save_path(FLAGS_save_dir, FLAGS_image); cv::imwrite(save_path, vis_img); result.clear(); std::cout << "Visualized output saved as " << save_path << std::endl; } - std::cout << "Total running time: " << total_running_time_s - << " s, average running time: " << total_running_time_s / imgs - << " s/img, total read img time: " << total_imread_time_s - << " s, average read img time: " << total_imread_time_s / imgs - << " s, batch_size = " << FLAGS_batch_size << std::endl; - return 0; } diff --git a/deploy/cpp/include/paddlex/transforms.h b/deploy/cpp/include/paddlex/transforms.h index c1ffd7e1de8a28f88a571e7b9d029585806cf59d..7e936dc17f4b6e58cdb8cdc36639173ccc24177c 100644 --- a/deploy/cpp/include/paddlex/transforms.h +++ b/deploy/cpp/include/paddlex/transforms.h @@ -214,6 +214,12 @@ class Padding : public Transform { height_ = item["target_size"].as>()[1]; } } + if (item["im_padding_value"].IsDefined()) { + im_value_ = item["im_padding_value"].as>(); + } + else { + im_value_ = {0, 0, 0}; + } } virtual bool Run(cv::Mat* im, ImageBlob* data); @@ -221,6 +227,7 @@ class Padding : public Transform { int coarsest_stride_ = -1; int width_ = 0; int height_ = 0; + std::vector im_value_; }; /* * @brief diff --git a/deploy/cpp/include/paddlex/visualize.h b/deploy/cpp/include/paddlex/visualize.h index 9ddba5387b427c60645db7c96a54bcba76fa9898..9b80ca367bc8e45334c951cb6dd32069c67c9dbd 100644 --- a/deploy/cpp/include/paddlex/visualize.h +++ b/deploy/cpp/include/paddlex/visualize.h @@ -65,13 +65,12 @@ std::vector GenerateColorMap(int num_class); * @param img: initial image matrix * @param results: the detection result * @param labels: label map - * @param colormap: visualization color map + * @param threshold: minimum confidence to display * @return visualized image matrix * */ cv::Mat Visualize(const cv::Mat& img, const DetResult& results, const std::map& labels, - const std::vector& colormap, float threshold = 0.5); /* @@ -81,13 +80,11 @@ cv::Mat Visualize(const cv::Mat& img, * @param img: initial image matrix * @param results: the detection result * @param labels: label map - * @param colormap: visualization color map * @return visualized image matrix * */ cv::Mat Visualize(const cv::Mat& img, const SegResult& result, - const std::map& labels, - const std::vector& colormap); + const std::map& labels); /* * @brief diff --git a/deploy/cpp/src/transforms.cpp b/deploy/cpp/src/transforms.cpp index 99a73ee7345bbc8cc672d1c42627a9326ded0cf7..626b2053d2473bcf66fcb1a760d9ce2e101324f4 100644 --- a/deploy/cpp/src/transforms.cpp +++ b/deploy/cpp/src/transforms.cpp @@ -110,8 +110,9 @@ bool Padding::Run(cv::Mat* im, ImageBlob* data) { << ", but they should be greater than 0." << std::endl; return false; } + cv::Scalar value = cv::Scalar(im_value_[0], im_value_[1], im_value_[2]); cv::copyMakeBorder( - *im, *im, 0, padding_h, 0, padding_w, cv::BORDER_CONSTANT, cv::Scalar(0)); + *im, *im, 0, padding_h, 0, padding_w, cv::BORDER_CONSTANT, value); data->new_im_size_[0] = im->rows; data->new_im_size_[1] = im->cols; return true; diff --git a/deploy/cpp/src/visualize.cpp b/deploy/cpp/src/visualize.cpp index 1511887f097e20826f13c8c1f098ceea4efc0b5b..afc1733b497269b706bf4e07d82f3a7aa43087f5 100644 --- a/deploy/cpp/src/visualize.cpp +++ b/deploy/cpp/src/visualize.cpp @@ -34,8 +34,8 @@ std::vector GenerateColorMap(int num_class) { cv::Mat Visualize(const cv::Mat& img, const DetResult& result, const std::map& labels, - const std::vector& colormap, float threshold) { + auto colormap = GenerateColorMap(labels.size()); cv::Mat vis_img = img.clone(); auto boxes = result.boxes; for (int i = 0; i < boxes.size(); ++i) { @@ -107,8 +107,8 @@ cv::Mat Visualize(const cv::Mat& img, cv::Mat Visualize(const cv::Mat& img, const SegResult& result, - const std::map& labels, - const std::vector& colormap) { + const std::map& labels) { + auto colormap = GenerateColorMap(labels.size()); std::vector label_map(result.label_map.data.begin(), result.label_map.data.end()); cv::Mat mask(result.label_map.shape[0], diff --git a/deploy/lite/android/demo/.gitignore b/deploy/lite/android/demo/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..2b75303ac58f551de0a327638a60b909c6d33ece --- /dev/null +++ b/deploy/lite/android/demo/.gitignore @@ -0,0 +1,13 @@ +*.iml +.gradle +/local.properties +/.idea/caches +/.idea/libraries +/.idea/modules.xml +/.idea/workspace.xml +/.idea/navEditor.xml +/.idea/assetWizardSettings.xml +.DS_Store +/build +/captures +.externalNativeBuild diff --git a/deploy/lite/android/demo/app/.gitignore b/deploy/lite/android/demo/app/.gitignore new file mode 100644 index 0000000000000000000000000000000000000000..796b96d1c402326528b4ba3c12ee9d92d0e212e9 --- /dev/null +++ b/deploy/lite/android/demo/app/.gitignore @@ -0,0 +1 @@ +/build diff --git a/deploy/lite/android/demo/app/build.gradle b/deploy/lite/android/demo/app/build.gradle new file mode 100644 index 0000000000000000000000000000000000000000..f743f1d23905566772c4e572e9700df5ad779ca0 --- /dev/null +++ b/deploy/lite/android/demo/app/build.gradle @@ -0,0 +1,119 @@ +import java.security.MessageDigest + +apply plugin: 'com.android.application' + +android { + compileSdkVersion 28 + defaultConfig { + applicationId "com.baidu.paddlex.lite.demo" + minSdkVersion 15 + targetSdkVersion 28 + versionCode 1 + versionName "1.0" + testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" + } + buildTypes { + release { + minifyEnabled false + proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' + } + } +} + +dependencies { + implementation fileTree(include: ['*.aar'], dir: 'libs') + implementation 'com.android.support:appcompat-v7:28.0.0' + implementation 'com.android.support.constraint:constraint-layout:1.1.3' + implementation 'com.android.support:design:28.0.0' + testImplementation 'junit:junit:4.12' + androidTestImplementation 'com.android.support.test:runner:1.0.2' + androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2' +} + + +def paddlexAndroidSdk = 'https://bj.bcebos.com/paddlex/deploy/lite/paddlex_lite_11cbd50e.tar.gz' + +task downloadAndExtractPaddleXAndroidSdk(type: DefaultTask) { + doFirst { + println "Downloading and extracting PaddleX Android SDK"} + doLast { + // Prepare cache folder for sdk + if (!file("cache").exists()) { + mkdir "cache" + } + // Generate cache name for sdk + MessageDigest messageDigest = MessageDigest.getInstance('MD5') + messageDigest.update(paddlexAndroidSdk.bytes) + String cacheName = new BigInteger(1, messageDigest.digest()).toString(32) + // Download sdk + if (!file("cache/${cacheName}.tar.gz").exists()) { + ant.get(src: paddlexAndroidSdk, dest: file("cache/${cacheName}.tar.gz")) + } + // Unpack sdk + copy { + from tarTree("cache/${cacheName}.tar.gz") + into "cache/${cacheName}" + } + // Copy sdk + if (!file("libs/paddlex.aar").exists()) { + copy { + from "cache/${cacheName}/paddlex.aar" + into "libs" + } + } + } +} + +preBuild.dependsOn downloadAndExtractPaddleXAndroidSdk + +def paddleXLiteModel = 'https://bj.bcebos.com/paddlex/deploy/lite/mobilenetv2_imagenet_lite2.6.1.tar.gz' +task downloadAndExtractPaddleXLiteModel(type: DefaultTask) { + doFirst { + println "Downloading and extracting PaddleX Android SDK"} + + doLast { + // Prepare cache folder for model + if (!file("cache").exists()) { + mkdir "cache" + } + // Generate cache name for model + MessageDigest messageDigest = MessageDigest.getInstance('MD5') + messageDigest.update(paddleXLiteModel.bytes) + String cacheName = new BigInteger(1, messageDigest.digest()).toString(32) + // Download sdk + if (!file("cache/${cacheName}.tar.gz").exists()) { + ant.get(src: paddleXLiteModel, dest: file("cache/${cacheName}.tar.gz")) + } + + // Unpack model + copy { + from tarTree("cache/${cacheName}.tar.gz") + into "cache/${cacheName}" + } + + // Copy model.nb + if (!file("src/main/assets/model/model.nb").exists()) { + copy { + from "cache/${cacheName}/model.nb" + into "src/main/assets/model/" + } + } + // Copy config file model.yml + if (!file("src/main/assets/config/model.yml").exists()) { + copy { + from "cache/${cacheName}/model.yml" + into "src/main/assets/config/" + } + } + // Copy config file model.yml + if (!file("src/main/assets/images/test.jpg").exists()) { + copy { + from "cache/${cacheName}/test.jpg" + into "src/main/assets/images/" + } + } + } + +} + +preBuild.dependsOn downloadAndExtractPaddleXLiteModel diff --git a/deploy/lite/android/demo/app/proguard-rules.pro b/deploy/lite/android/demo/app/proguard-rules.pro new file mode 100644 index 0000000000000000000000000000000000000000..f1b424510da51fd82143bc74a0a801ae5a1e2fcd --- /dev/null +++ b/deploy/lite/android/demo/app/proguard-rules.pro @@ -0,0 +1,21 @@ +# Add project specific ProGuard rules here. +# You can control the set of applied configuration files using the +# proguardFiles setting in build.gradle. +# +# For more details, see +# http://developer.android.com/guide/developing/tools/proguard.html + +# If your project uses WebView with JS, uncomment the following +# and specify the fully qualified class name to the JavaScript interface +# class: +#-keepclassmembers class fqcn.of.javascript.interface.for.webview { +# public *; +#} + +# Uncomment this to preserve the line number information for +# debugging stack traces. +#-keepattributes SourceFile,LineNumberTable + +# If you keep the line number information, uncomment this to +# hide the original source file name. +#-renamesourcefileattribute SourceFile diff --git a/deploy/lite/android/demo/app/src/androidTest/java/com/baidu/paddlex/lite/demo/ExampleInstrumentedTest.java b/deploy/lite/android/demo/app/src/androidTest/java/com/baidu/paddlex/lite/demo/ExampleInstrumentedTest.java new file mode 100644 index 0000000000000000000000000000000000000000..4b58dec6f5dd8bfa083ec951d659dd0690f67221 --- /dev/null +++ b/deploy/lite/android/demo/app/src/androidTest/java/com/baidu/paddlex/lite/demo/ExampleInstrumentedTest.java @@ -0,0 +1,32 @@ +package com.baidu.paddlex.lite.demo; + +import android.content.Context; +import android.content.res.AssetManager; +import android.support.test.InstrumentationRegistry; +import android.support.test.runner.AndroidJUnit4; + +import com.baidu.paddlex.config.ConfigParser; + +import org.junit.Test; +import org.junit.runner.RunWith; + +import java.io.IOException; +import java.io.InputStream; + +import static org.junit.Assert.assertEquals; + +/** + * Instrumented test, which will execute on an Android device. + * + * @see Testing documentation + */ +@RunWith(AndroidJUnit4.class) +public class ExampleInstrumentedTest { + @Test + public void useAppContext() throws IOException { + // Context of the app under test. + Context appContext = InstrumentationRegistry.getTargetContext(); + AssetManager ass = appContext.getAssets(); + assertEquals("com.baidu.paddlex.lite.demo", appContext.getPackageName()); + } +} diff --git a/deploy/lite/android/demo/app/src/main/AndroidManifest.xml b/deploy/lite/android/demo/app/src/main/AndroidManifest.xml new file mode 100644 index 0000000000000000000000000000000000000000..940c9692fcf6fdfe6b07e8f4641fe7e9a9e5ff5f --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/AndroidManifest.xml @@ -0,0 +1,28 @@ + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/AppCompatPreferenceActivity.java b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/AppCompatPreferenceActivity.java new file mode 100644 index 0000000000000000000000000000000000000000..c6f4eff8e736278c71ef2c34783dd3e1b3659495 --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/AppCompatPreferenceActivity.java @@ -0,0 +1,126 @@ +// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.baidu.paddlex.lite.demo; + +import android.content.res.Configuration; +import android.os.Bundle; +import android.preference.PreferenceActivity; +import android.support.annotation.LayoutRes; +import android.support.annotation.Nullable; +import android.support.v7.app.ActionBar; +import android.support.v7.app.AppCompatDelegate; +import android.support.v7.widget.Toolbar; +import android.view.MenuInflater; +import android.view.View; +import android.view.ViewGroup; + +/** + * A {@link android.preference.PreferenceActivity} which implements and proxies the necessary calls + * to be used with AppCompat. + *

+ * This technique can be used with an {@link android.app.Activity} class, not just + * {@link android.preference.PreferenceActivity}. + */ + +public abstract class AppCompatPreferenceActivity extends PreferenceActivity { + private AppCompatDelegate mDelegate; + + @Override + protected void onCreate(Bundle savedInstanceState) { + getDelegate().installViewFactory(); + getDelegate().onCreate(savedInstanceState); + super.onCreate(savedInstanceState); + } + + @Override + protected void onPostCreate(Bundle savedInstanceState) { + super.onPostCreate(savedInstanceState); + getDelegate().onPostCreate(savedInstanceState); + } + + public ActionBar getSupportActionBar() { + return getDelegate().getSupportActionBar(); + } + + public void setSupportActionBar(@Nullable Toolbar toolbar) { + getDelegate().setSupportActionBar(toolbar); + } + + @Override + public MenuInflater getMenuInflater() { + return getDelegate().getMenuInflater(); + } + + @Override + public void setContentView(@LayoutRes int layoutResID) { + getDelegate().setContentView(layoutResID); + } + + @Override + public void setContentView(View view) { + getDelegate().setContentView(view); + } + + @Override + public void setContentView(View view, ViewGroup.LayoutParams params) { + getDelegate().setContentView(view, params); + } + + @Override + public void addContentView(View view, ViewGroup.LayoutParams params) { + getDelegate().addContentView(view, params); + } + + @Override + protected void onPostResume() { + super.onPostResume(); + getDelegate().onPostResume(); + } + + @Override + protected void onTitleChanged(CharSequence title, int color) { + super.onTitleChanged(title, color); + getDelegate().setTitle(title); + } + + @Override + public void onConfigurationChanged(Configuration newConfig) { + super.onConfigurationChanged(newConfig); + getDelegate().onConfigurationChanged(newConfig); + } + + @Override + protected void onStop() { + super.onStop(); + getDelegate().onStop(); + } + + @Override + protected void onDestroy() { + super.onDestroy(); + getDelegate().onDestroy(); + } + + public void invalidateOptionsMenu() { + getDelegate().invalidateOptionsMenu(); + } + + private AppCompatDelegate getDelegate() { + if (mDelegate == null) { + mDelegate = AppCompatDelegate.create(this, null); + } + return mDelegate; + } +} diff --git a/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/MainActivity.java b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/MainActivity.java new file mode 100644 index 0000000000000000000000000000000000000000..62e47214fc80a40fbfa173967f61e490eab92e47 --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/MainActivity.java @@ -0,0 +1,466 @@ +// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.baidu.paddlex.lite.demo; + +import android.Manifest; +import android.app.ProgressDialog; +import android.content.ContentResolver; +import android.content.Intent; +import android.content.SharedPreferences; +import android.content.pm.PackageManager; +import android.database.Cursor; +import android.graphics.Bitmap; +import android.graphics.BitmapFactory; +import android.net.Uri; +import android.os.Bundle; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Message; +import android.preference.PreferenceManager; +import android.provider.MediaStore; +import android.support.annotation.NonNull; +import android.support.v4.app.ActivityCompat; +import android.support.v4.content.ContextCompat; +import android.support.v7.app.AppCompatActivity; +import android.text.method.ScrollingMovementMethod; +import android.util.Log; +import android.view.Menu; +import android.view.MenuInflater; +import android.view.MenuItem; +import android.view.View; +import android.widget.Button; +import android.widget.ImageView; +import android.widget.TextView; +import android.widget.Toast; +import com.baidu.paddlex.Predictor; +import com.baidu.paddlex.Utils; +import com.baidu.paddlex.config.ConfigParser; +import com.baidu.paddlex.postprocess.ClsResult; +import com.baidu.paddlex.postprocess.DetResult; +import com.baidu.paddlex.postprocess.SegResult; +import com.baidu.paddlex.visual.Visualize; +import org.opencv.core.Mat; +import org.opencv.imgcodecs.Imgcodecs; +import org.opencv.imgproc.Imgproc; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; + +public class MainActivity extends AppCompatActivity { + public static final int OPEN_GALLERY_REQUEST_CODE = 0; + public static final int TAKE_PHOTO_REQUEST_CODE = 1; + public static final int REQUEST_LOAD_MODEL = 0; + public static final int REQUEST_RUN_MODEL = 1; + public static final int RESPONSE_LOAD_MODEL_SUCCESSED = 0; + public static final int RESPONSE_LOAD_MODEL_FAILED = 1; + public static final int RESPONSE_RUN_MODEL_SUCCESSED = 2; + public static final int RESPONSE_RUN_MODEL_FAILED = 3; + private static final String TAG = MainActivity.class.getSimpleName(); + protected ProgressDialog pbLoadModel = null; + protected ProgressDialog pbRunModel = null; + + protected Handler receiver = null; // receive messages from worker thread + protected Handler sender = null; // send command to worker thread + protected HandlerThread worker = null; // worker thread to load&run model + + protected TextView tvInputSetting; + protected ImageView ivInputImage; + protected TextView tvOutputResult; + protected TextView tvInferenceTime; + private Button predictButton; + protected String testImagePathFromAsset; + protected String testYamlPathFromAsset; + protected String testModelPathFromAsset; + + // Predictor + protected Predictor predictor = new Predictor(); + // model config + protected ConfigParser configParser = new ConfigParser(); + // Visualize + protected Visualize visualize = new Visualize(); + // Predict Mat of Opencv + protected Mat predictMat; + + + + + @Override + protected void onCreate(Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + setContentView(R.layout.activity_main); + receiver = new Handler() { + @Override + public void handleMessage(Message msg) { + switch (msg.what) { + case RESPONSE_LOAD_MODEL_SUCCESSED: + pbLoadModel.dismiss(); + Toast.makeText(MainActivity.this, "Load model successfully!", Toast.LENGTH_SHORT).show(); + break; + case RESPONSE_LOAD_MODEL_FAILED: + pbLoadModel.dismiss(); + Toast.makeText(MainActivity.this, "Load model failed!", Toast.LENGTH_SHORT).show(); + break; + case RESPONSE_RUN_MODEL_SUCCESSED: + pbRunModel.dismiss(); + onRunModelSuccessed(); + break; + case RESPONSE_RUN_MODEL_FAILED: + pbRunModel.dismiss(); + Toast.makeText(MainActivity.this, "Run model failed!", Toast.LENGTH_SHORT).show(); + onRunModelFailed(); + break; + default: + break; + } + } + }; + worker = new HandlerThread("Predictor Worker"); + worker.start(); + sender = new Handler(worker.getLooper()) { + public void handleMessage(Message msg) { + switch (msg.what) { + case REQUEST_LOAD_MODEL: + // load model and reload test image + if (onLoadModel()) { + receiver.sendEmptyMessage(RESPONSE_LOAD_MODEL_SUCCESSED); + } else { + receiver.sendEmptyMessage(RESPONSE_LOAD_MODEL_FAILED); + } + break; + case REQUEST_RUN_MODEL: + // run model if model is loaded + if (onRunModel()) { + receiver.sendEmptyMessage(RESPONSE_RUN_MODEL_SUCCESSED); + } else { + receiver.sendEmptyMessage(RESPONSE_RUN_MODEL_FAILED); + } + break; + default: + break; + } + } + }; + + tvInputSetting = findViewById(R.id.tv_input_setting); + ivInputImage = findViewById(R.id.iv_input_image); + predictButton = findViewById(R.id.iv_predict_button); + tvInferenceTime = findViewById(R.id.tv_inference_time); + tvOutputResult = findViewById(R.id.tv_output_result); + tvInputSetting.setMovementMethod(ScrollingMovementMethod.getInstance()); + tvOutputResult.setMovementMethod(ScrollingMovementMethod.getInstance()); + SharedPreferences sharedPreferences = PreferenceManager.getDefaultSharedPreferences(this); + String image_path = sharedPreferences.getString(getString(R.string.IMAGE_PATH_KEY), + getString(R.string.IMAGE_PATH_DEFAULT)); + Utils.initialOpencv(); + loadTestImageFromAsset(image_path); + predictButton.setOnClickListener(new View.OnClickListener() { + @Override + public void onClick(View v) { + if(predictor.isLoaded()){ + onLoadModelSuccessed(); + } + } + }); + + } + + public boolean onLoadModel() { + return predictor.init(configParser); + } + + public boolean onRunModel() { + return predictor.isLoaded() && predictor.predict(); + } + + public void onRunModelFailed() { + } + + public void loadModel() { + pbLoadModel = ProgressDialog.show(this, "", "Loading model...", false, false); + sender.sendEmptyMessage(REQUEST_LOAD_MODEL); + } + + public void runModel() { + pbRunModel = ProgressDialog.show(this, "", "Running model...", false, false); + sender.sendEmptyMessage(REQUEST_RUN_MODEL); + } + + public void onLoadModelSuccessed() { + if (predictMat != null && predictor.isLoaded()) { + int w = predictMat.width(); + int h = predictMat.height(); + int c = predictMat.channels(); + predictor.setInputMat(predictMat); + runModel(); + } + } + + public void onRunModelSuccessed() { + // obtain results and update UI + tvInferenceTime.setText("Inference time: " + predictor.getInferenceTime() + " ms"); + + if (configParser.getModelType().equalsIgnoreCase("segmenter")) { + SegResult segResult = predictor.getSegResult(); + Mat maskMat = visualize.draw(segResult, predictMat.clone(), predictor.getImageBlob(), 1); + Imgproc.cvtColor(maskMat, maskMat, Imgproc.COLOR_BGRA2RGBA); + Bitmap outputImage = Bitmap.createBitmap(maskMat.width(), maskMat.height(), Bitmap.Config.ARGB_8888); + org.opencv.android.Utils.matToBitmap(maskMat, outputImage); + if (outputImage != null) { + ivInputImage.setImageBitmap(outputImage); + } + } else if (configParser.getModelType().equalsIgnoreCase("detector")) { + DetResult detResult = predictor.getDetResult(); + Mat roiMat = visualize.draw(detResult, predictMat.clone()); + Imgproc.cvtColor(roiMat, roiMat, Imgproc.COLOR_BGR2RGB); + Bitmap outputImage = Bitmap.createBitmap(roiMat.width(),roiMat.height(), Bitmap.Config.ARGB_8888); + org.opencv.android.Utils.matToBitmap(roiMat,outputImage); + if (outputImage != null) { + ivInputImage.setImageBitmap(outputImage); + } + } else if (configParser.getModelType().equalsIgnoreCase("classifier")) { + ClsResult clsResult = predictor.getClsResult(); + if (configParser.getLabeList().size() > 0) { + String outputResult = "Top1: " + clsResult.getCategory() + " - " + String.format("%.3f", clsResult.getScore()); + tvOutputResult.setText(outputResult); + tvOutputResult.scrollTo(0, 0); + } + } + } + + public void onMatChanged(Mat mat) { + this.predictMat = mat.clone(); + } + + public void onImageChanged(Bitmap image) { + ivInputImage.setImageBitmap(image); + tvOutputResult.setText(""); + tvInferenceTime.setText("Inference time: -- ms"); + } + + public void onSettingsClicked() { + startActivity(new Intent(MainActivity.this, SettingsActivity.class)); + } + + @Override + public boolean onCreateOptionsMenu(Menu menu) { + MenuInflater inflater = getMenuInflater(); + inflater.inflate(R.menu.menu_action_options, menu); + return true; + } + + @Override + public boolean onOptionsItemSelected(MenuItem item) { + switch (item.getItemId()) { + case android.R.id.home: + finish(); + break; + case R.id.open_gallery: + if (requestAllPermissions()) { + openGallery(); + } + break; + case R.id.take_photo: + if (requestAllPermissions()) { + takePhoto(); + } + break; + case R.id.settings: + if (requestAllPermissions()) { + // make sure we have SDCard r&w permissions to load model from SDCard + onSettingsClicked(); + } + break; + } + return super.onOptionsItemSelected(item); + } + + @Override + public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, + @NonNull int[] grantResults) { + super.onRequestPermissionsResult(requestCode, permissions, grantResults); + if (grantResults[0] != PackageManager.PERMISSION_GRANTED || grantResults[1] != PackageManager.PERMISSION_GRANTED) { + Toast.makeText(this, "Permission Denied", Toast.LENGTH_SHORT).show(); + } + } + + @Override + protected void onActivityResult(int requestCode, int resultCode, Intent data) { + super.onActivityResult(requestCode, resultCode, data); + if (resultCode == RESULT_OK && data != null) { + switch (requestCode) { + case OPEN_GALLERY_REQUEST_CODE: + try { + ContentResolver resolver = getContentResolver(); + Uri uri = data.getData(); + Bitmap image = MediaStore.Images.Media.getBitmap(resolver, uri); + String[] proj = {MediaStore.Images.Media.DATA}; + Cursor cursor = managedQuery(uri, proj, null, null, null); + cursor.moveToFirst(); + int columnIndex = cursor.getColumnIndex(proj[0]); + String imgDecodableString = cursor.getString(columnIndex); + File file = new File(imgDecodableString); + Mat mat = Imgcodecs.imread(file.getAbsolutePath(),Imgcodecs.IMREAD_COLOR); + onImageChanged(image); + onMatChanged(mat); + } catch (IOException e) { + Log.e(TAG, e.toString()); + } + break; + case TAKE_PHOTO_REQUEST_CODE: + Bitmap image = (Bitmap) data.getParcelableExtra("data"); + Mat mat = new Mat(); + org.opencv.android.Utils.bitmapToMat(image, mat); + Imgproc.cvtColor(mat, mat, Imgproc.COLOR_RGBA2BGR); + onImageChanged(image); + onMatChanged(mat); + break; + default: + break; + } + } + } + + private boolean requestAllPermissions() { + if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) + != PackageManager.PERMISSION_GRANTED || ContextCompat.checkSelfPermission(this, + Manifest.permission.CAMERA) + != PackageManager.PERMISSION_GRANTED) { + ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE, + Manifest.permission.CAMERA}, + 0); + return false; + } + return true; + } + + private void openGallery() { + Intent intent = new Intent(Intent.ACTION_PICK, null); + intent.setDataAndType(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, "image/*"); + startActivityForResult(intent, OPEN_GALLERY_REQUEST_CODE); + } + + private void takePhoto() { + Intent takePhotoIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE); + if (takePhotoIntent.resolveActivity(getPackageManager()) != null) { + startActivityForResult(takePhotoIntent, TAKE_PHOTO_REQUEST_CODE); + } + } + + @Override + public boolean onPrepareOptionsMenu(Menu menu) { + boolean isLoaded = predictor.isLoaded(); + menu.findItem(R.id.open_gallery).setEnabled(isLoaded); + menu.findItem(R.id.take_photo).setEnabled(isLoaded); + return super.onPrepareOptionsMenu(menu); + } + + @Override + protected void onResume() { + Log.i(TAG, "begin onResume"); + super.onResume(); + SharedPreferences sharedPreferences = PreferenceManager.getDefaultSharedPreferences(this); + + boolean settingsChanged = false; + boolean testImageChanged = false; + String modelPath = sharedPreferences.getString(getString(R.string.MODEL_PATH_KEY), + getString(R.string.MODEL_PATH_DEFAULT)); + settingsChanged |= !modelPath.equalsIgnoreCase(testModelPathFromAsset); + String yamlPath = sharedPreferences.getString(getString(R.string.YAML_PATH_KEY), + getString(R.string.YAML_PATH_DEFAULT)); + settingsChanged |= !yamlPath.equalsIgnoreCase(testYamlPathFromAsset); + int cpuThreadNum = Integer.parseInt(sharedPreferences.getString(getString(R.string.CPU_THREAD_NUM_KEY), + getString(R.string.CPU_THREAD_NUM_DEFAULT))); + settingsChanged |= cpuThreadNum != configParser.getCpuThreadNum(); + String cpuPowerMode = sharedPreferences.getString(getString(R.string.CPU_POWER_MODE_KEY), + getString(R.string.CPU_POWER_MODE_DEFAULT)); + settingsChanged |= !cpuPowerMode.equalsIgnoreCase(configParser.getCpuPowerMode()); + String imagePath = sharedPreferences.getString(getString(R.string.IMAGE_PATH_KEY), + getString(R.string.IMAGE_PATH_DEFAULT)); + testImageChanged |= !imagePath.equalsIgnoreCase(testImagePathFromAsset); + + testYamlPathFromAsset = yamlPath; + testModelPathFromAsset = modelPath; + if (settingsChanged) { + try { + String realModelPath = modelPath; + if (!modelPath.substring(0, 1).equals("/")) { + String modelFileName = Utils.getFileNameFromString(modelPath); + realModelPath = this.getCacheDir() + File.separator + modelFileName; + Utils.copyFileFromAssets(this, modelPath, realModelPath); + } + String realYamlPath = yamlPath; + if (!yamlPath.substring(0, 1).equals("/")) { + String yamlFileName = Utils.getFileNameFromString(yamlPath); + realYamlPath = this.getCacheDir() + File.separator + yamlFileName; + Utils.copyFileFromAssets(this, yamlPath, realYamlPath); + } + configParser.init(realModelPath, realYamlPath, cpuThreadNum, cpuPowerMode); + visualize.init(configParser.getNumClasses()); + } catch (IOException e) { + e.printStackTrace(); + Toast.makeText(MainActivity.this, "Load config failed!", Toast.LENGTH_SHORT).show(); + } + // update UI + tvInputSetting.setText("Model: " + configParser.getModel()+ "\n" + "CPU" + + " Thread Num: " + Integer.toString(configParser.getCpuThreadNum()) + "\n" + "CPU Power Mode: " + configParser.getCpuPowerMode()); + tvInputSetting.scrollTo(0, 0); + // reload model if configure has been changed + loadModel(); + } + + if (testImageChanged){ + loadTestImageFromAsset(imagePath); + } + } + + public void loadTestImageFromAsset(String imagePath){ + if (imagePath.isEmpty()) { + return; + } + // read test image file from custom file_paths if the first character of mode file_paths is '/', otherwise read test + // image file from assets + testImagePathFromAsset = imagePath; + if (!imagePath.substring(0, 1).equals("/")) { + InputStream imageStream = null; + try { + imageStream = getAssets().open(imagePath); + } catch (IOException e) { + e.printStackTrace(); + } + onImageChanged(BitmapFactory.decodeStream(imageStream)); + String realPath; + String imageFileName = Utils.getFileNameFromString(imagePath); + realPath = this.getCacheDir() + File.separator + imageFileName; + Utils.copyFileFromAssets(this, imagePath, realPath); + onMatChanged(Imgcodecs.imread(realPath, Imgcodecs.IMREAD_COLOR)); + } else { + if (!new File(imagePath).exists()) { + return; + } + onMatChanged(Imgcodecs.imread(imagePath, Imgcodecs.IMREAD_COLOR)); + onImageChanged( BitmapFactory.decodeFile(imagePath)); + } + } + + @Override + protected void onDestroy() { + if (predictor != null) { + predictor.releaseModel(); + } + worker.quit(); + super.onDestroy(); + } +} \ No newline at end of file diff --git a/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/SettingsActivity.java b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/SettingsActivity.java new file mode 100644 index 0000000000000000000000000000000000000000..271343ff5a626ba5d8a224dfe832738ae4ede123 --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/java/com/baidu/paddlex/lite/demo/SettingsActivity.java @@ -0,0 +1,158 @@ +// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package com.baidu.paddlex.lite.demo; + +import com.baidu.paddlex.Utils; + +import android.content.SharedPreferences; +import android.os.Bundle; +import android.preference.CheckBoxPreference; +import android.preference.EditTextPreference; +import android.preference.ListPreference; +import android.support.v7.app.ActionBar; + +import java.util.ArrayList; +import java.util.List; + +public class SettingsActivity extends AppCompatPreferenceActivity implements SharedPreferences.OnSharedPreferenceChangeListener { + ListPreference lpChoosePreInstalledModel = null; + CheckBoxPreference cbEnableCustomSettings = null; + EditTextPreference etModelPath = null; + EditTextPreference etYamlPath = null; + EditTextPreference etImagePath = null; + ListPreference lpCPUThreadNum = null; + ListPreference lpCPUPowerMode = null; + + List preInstalledModelPaths = null; + List preInstalledYamlPaths = null; + List preInstalledImagePaths = null; + List preInstalledCPUThreadNums = null; + List preInstalledCPUPowerModes = null; + + @Override + public void onCreate(Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + addPreferencesFromResource(R.xml.settings); + ActionBar supportActionBar = getSupportActionBar(); + if (supportActionBar != null) { + supportActionBar.setDisplayHomeAsUpEnabled(true); + } + + // initialized pre-installed models + preInstalledModelPaths = new ArrayList(); + preInstalledYamlPaths = new ArrayList(); + preInstalledImagePaths = new ArrayList(); + preInstalledCPUThreadNums = new ArrayList(); + preInstalledCPUPowerModes = new ArrayList(); + preInstalledModelPaths.add(getString(R.string.MODEL_PATH_DEFAULT)); + preInstalledYamlPaths.add(getString(R.string.YAML_PATH_DEFAULT)); + preInstalledImagePaths.add(getString(R.string.IMAGE_PATH_DEFAULT)); + preInstalledCPUThreadNums.add(getString(R.string.CPU_THREAD_NUM_DEFAULT)); + preInstalledCPUPowerModes.add(getString(R.string.CPU_POWER_MODE_DEFAULT)); + // initialize UI components + lpChoosePreInstalledModel = + (ListPreference) findPreference(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY)); + String[] preInstalledModelNames = new String[preInstalledModelPaths.size()]; + for (int i = 0; i < preInstalledModelPaths.size(); i++) { + preInstalledModelNames[i] = + preInstalledModelPaths.get(i).substring(preInstalledModelPaths.get(i).lastIndexOf("/") + 1); + } + lpChoosePreInstalledModel.setEntries(preInstalledModelNames); + lpChoosePreInstalledModel.setEntryValues(preInstalledModelPaths.toArray(new String[preInstalledModelPaths.size()])); + cbEnableCustomSettings = + (CheckBoxPreference) findPreference(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY)); + etModelPath = (EditTextPreference) findPreference(getString(R.string.MODEL_PATH_KEY)); + etModelPath.setTitle("Model Path (SDCard: " + Utils.getSDCardDirectory() + ")"); + etYamlPath = (EditTextPreference) findPreference(getString(R.string.YAML_PATH_KEY)); + etImagePath = (EditTextPreference) findPreference(getString(R.string.IMAGE_PATH_KEY)); + lpCPUThreadNum = + (ListPreference) findPreference(getString(R.string.CPU_THREAD_NUM_KEY)); + lpCPUPowerMode = + (ListPreference) findPreference(getString(R.string.CPU_POWER_MODE_KEY)); + } + + private void reloadPreferenceAndUpdateUI() { + SharedPreferences sharedPreferences = getPreferenceScreen().getSharedPreferences(); + boolean enableCustomSettings = + sharedPreferences.getBoolean(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY), false); + String modelPath = sharedPreferences.getString(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY), + getString(R.string.MODEL_PATH_DEFAULT)); + int modelIdx = lpChoosePreInstalledModel.findIndexOfValue(modelPath); + if (modelIdx >= 0 && modelIdx < preInstalledModelPaths.size()) { + if (!enableCustomSettings) { + SharedPreferences.Editor editor = sharedPreferences.edit(); + editor.putString(getString(R.string.MODEL_PATH_KEY), preInstalledModelPaths.get(modelIdx)); + editor.putString(getString(R.string.YAML_PATH_KEY), preInstalledYamlPaths.get(modelIdx)); + editor.putString(getString(R.string.IMAGE_PATH_KEY), preInstalledImagePaths.get(modelIdx)); + editor.putString(getString(R.string.CPU_THREAD_NUM_KEY), preInstalledCPUThreadNums.get(modelIdx)); + editor.putString(getString(R.string.CPU_POWER_MODE_KEY), preInstalledCPUPowerModes.get(modelIdx)); + editor.commit(); + } + lpChoosePreInstalledModel.setSummary(modelPath); + } + + cbEnableCustomSettings.setChecked(enableCustomSettings); + etModelPath.setEnabled(enableCustomSettings); + etYamlPath.setEnabled(enableCustomSettings); + etImagePath.setEnabled(enableCustomSettings); + lpCPUThreadNum.setEnabled(enableCustomSettings); + lpCPUPowerMode.setEnabled(enableCustomSettings); + modelPath = sharedPreferences.getString(getString(R.string.MODEL_PATH_KEY), + getString(R.string.MODEL_PATH_DEFAULT)); + String YamlPath = sharedPreferences.getString(getString(R.string.YAML_PATH_KEY), + getString(R.string.YAML_PATH_DEFAULT)); + String imagePath = sharedPreferences.getString(getString(R.string.IMAGE_PATH_KEY), + getString(R.string.IMAGE_PATH_DEFAULT)); + String cpuThreadNum = sharedPreferences.getString(getString(R.string.CPU_THREAD_NUM_KEY), + getString(R.string.CPU_THREAD_NUM_DEFAULT)); + String cpuPowerMode = sharedPreferences.getString(getString(R.string.CPU_POWER_MODE_KEY), + getString(R.string.CPU_POWER_MODE_DEFAULT)); + + etModelPath.setSummary(modelPath); + etModelPath.setText(modelPath); + etYamlPath.setSummary(YamlPath); + etYamlPath.setText(YamlPath); + etImagePath.setSummary(imagePath); + etImagePath.setText(imagePath); + lpCPUThreadNum.setValue(cpuThreadNum); + lpCPUThreadNum.setSummary(cpuThreadNum); + lpCPUPowerMode.setValue(cpuPowerMode); + lpCPUPowerMode.setSummary(cpuPowerMode); + + } + + @Override + protected void onResume() { + super.onResume(); + getPreferenceScreen().getSharedPreferences().registerOnSharedPreferenceChangeListener(this); + reloadPreferenceAndUpdateUI(); + } + + @Override + protected void onPause() { + super.onPause(); + getPreferenceScreen().getSharedPreferences().unregisterOnSharedPreferenceChangeListener(this); + } + + @Override + public void onSharedPreferenceChanged(SharedPreferences sharedPreferences, String key) { + if (key.equals(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY))) { + SharedPreferences.Editor editor = sharedPreferences.edit(); + editor.putBoolean(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY), false); + editor.commit(); + } + reloadPreferenceAndUpdateUI(); + } +} diff --git a/deploy/lite/android/demo/app/src/main/res/drawable-v24/ic_launcher_foreground.xml b/deploy/lite/android/demo/app/src/main/res/drawable-v24/ic_launcher_foreground.xml new file mode 100644 index 0000000000000000000000000000000000000000..1f6bb290603d7caa16c5fb6f61bbfdc750622f5c --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/res/drawable-v24/ic_launcher_foreground.xml @@ -0,0 +1,34 @@ + + + + + + + + + + + diff --git a/deploy/lite/android/demo/app/src/main/res/drawable/face.jpg b/deploy/lite/android/demo/app/src/main/res/drawable/face.jpg new file mode 100644 index 0000000000000000000000000000000000000000..8963ae3db05894cd4bf3ea17957297363db73171 Binary files /dev/null and b/deploy/lite/android/demo/app/src/main/res/drawable/face.jpg differ diff --git a/deploy/lite/android/demo/app/src/main/res/drawable/ic_launcher_background.xml b/deploy/lite/android/demo/app/src/main/res/drawable/ic_launcher_background.xml new file mode 100644 index 0000000000000000000000000000000000000000..0d025f9bf6b67c63044a36a9ff44fbc69e5c5822 --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/res/drawable/ic_launcher_background.xml @@ -0,0 +1,170 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/deploy/lite/android/demo/app/src/main/res/layout/activity_main.xml b/deploy/lite/android/demo/app/src/main/res/layout/activity_main.xml new file mode 100644 index 0000000000000000000000000000000000000000..97c79f86dbedee3b71ef4b787b05352f70a428fd --- /dev/null +++ b/deploy/lite/android/demo/app/src/main/res/layout/activity_main.xml @@ -0,0 +1,112 @@ + + + + + + + + + + + + + + + + + + + + + + + +