Skip to content

P
PaddleHub

PaddlePaddle / PaddleHub
1 年多 前同步成功

通知 283
Star 12117
Fork 2091
P
PaddleHub
提交 d9f91206 编写于 作者: W wuzewu
浏览文件
操作

Merge branch 'release/v2.1' into develop

上级 e6b082e1
展开全部 隐藏空白更改
内联 并排
Showing with 0 addition and 8202 deletion
docs/imgs/output_18_2.png
查看替换文件 @ e6b082e1 浏览文件 @ d9f91206
docs/imgs/output_18_2.png

217.7 KB | W: | H:

docs/imgs/output_18_2.png

5.8 MB | W: | H:

docs/imgs/output_18_2.png
docs/imgs/output_18_2.png
docs/imgs/output_18_2.png
docs/imgs/output_18_2.png
  • 2-up
  • Swipe
  • Onion skin
docs/index.rst 已删除 100644 → 0
浏览文件 @ e6b082e1
#########################################
PaddleHub 文档
#########################################
.. toctree::
:caption: 目录
:maxdepth: 2
:titlesonly:
概述<overview>
安装<install>
快速体验<quick_experience/quick_index>
教程<tutorial/tutorial_index>
API<reference/ref_index>
FAQ<faq>
社区贡献<contribution/contri_index>
更新历史<release>
\ No newline at end of file
docs/install.md 已删除 100644 → 0
浏览文件 @ e6b082e1
# PaddleHub安装
## 环境准备
PaddleHub需要与飞桨一起使用，其硬件和操作系统的适用范围与[飞桨](https://www.paddlepaddle.org.cn/install/quick)相同。
> 注意：飞桨版本需要>= 1.7.0。
```python
# 查看是否安装飞桨
$python # 进入python解释器
import paddle.fluid
paddle.fluid.install_check.run_check()
```
> 如果出现`Your Paddle Fluid is installed successfully`，说明飞桨已成功安装。
```python
$pip list | grep paddlepaddle # 查看飞桨版本。pip list查看所有的package版本，grep负责根据关键字筛选。
```
## 安装操作
根据实际需要，执行以下命令之一进行PaddleHub的安装（推荐使用第一个）。
> 1.安装过程中需要网络连接，请确保机器可以正常访问网络。成功安装之后，可以离线使用。
2.如果已安装PaddleHub，再次执行安装操作将先卸载再安装。安装方式支持：安装指定版本和安装最新版本。
3.由于国内网速的问题，直接pip安装包通常速度非常慢，而且经常会出现装到一半失败了的问题。使用国内镜像可以节省时间，提高pip安装的效率。
```
国内镜像源列表：
清华大学：https://pypi.tuna.tsinghua.edu.cn/simple/
百度：https://mirror.baidu.com/pypi/simple
```
```python
$pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple # 安装最新版本，使用清华源
```
```python
$pip install paddlehub==1.6.1 -i https://pypi.tuna.tsinghua.edu.cn/simple # 安装指定版本（==1.6.1表示PaddleHub的版本），使用清华源
```
```python
$pip install paddlehub --upgrade -i https://mirror.baidu.com/pypi/simple # 安装最新版本，使用百度源
```
```python
$pip install paddlehub==1.6.1 -i https://mirror.baidu.com/pypi/simple # 安装指定版本（==1.6.1表示PaddleHub的版本），使用百度源
```
> 如果出现`Successfully installed paddlehub`，说明PaddleHub安装成功。
## 验证安装
检查PaddleHub是否安装成功。
```python
$pip list | grep paddlehub # pip list查看所有的package版本，grep负责根据关键字筛选
```
```python
$pip show paddlehub # 查看PaddleHub详细信息
```
PaddleHub详细信息的如下面所示，可以查看显示了PaddleHub的版本、位置等信息。
```
Name: paddlehub
Version: 1.7.1
Summary: A toolkit for managing pretrained models of PaddlePaddle and helping user getting started with transfer learning more efficiently.
Home-page: https://github.com/PaddlePaddle/PaddleHub
Author: PaddlePaddle Author
Author-email: paddle-dev@baidu.com
License: Apache 2.0
Location: /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages
Requires: pandas, pre-commit, gunicorn, flake8, visualdl, yapf, flask, protobuf, sentencepiece, six, cma, colorlog
Required-by:
```
## 如何卸载
此卸载仅卸载PaddleHub，已下载的模型文件和数据集仍保留。
```python
$pip uninstall paddlehub -y # 卸载PaddleHub
```
> 如果出现`Successfully uninstalled paddlehub`,表明PaddleHub卸载成功。
## 常见问题
1. 已安装PaddleHub，可以升级飞桨版本吗？
答复：可以。直接正常升级飞桨版本即可。
2. 已安装PaddleHub，如何升级？
答复：执行`pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple`，可以将PaddleHub升级到最新版本。
3. `upgrade`安装与安装指定版本有什么区别？
答复：`upgrade`安装的是最新版本，安装指定版本可以安装任意版本。
4. 如何设置PaddleHub下载的缓存位置？
答复：PaddleHub的Module默认会保存在用户目录下，可以通过修改环境变量`HUB_HOME`更改这个位置。
docs/quick_experience/cmd_quick_run.md 已删除 100644 → 0
浏览文件 @ e6b082e1
# 通过命令行调用方式使用PaddleHub
本页面的代码/命令可在[AIStudio](https://aistudio.baidu.com/aistudio/projectdetail/643120)上在线运行，类似notebook的环境，只需通过浏览器即可访问，无需准备环境，非常方便开发者快速体验。
PaddleHub在设计时，为模型的管理和使用提供了命令行工具，也提供了通过命令行调用PaddleHub模型完成预测的方式。比如，前面章节中人像分割和文本分词的任务也可以通过命令行调用的方式实现。
### 体验前请提前安装好PaddleHub
```python
# 指定版本安装PaddleHub，使用清华源更稳定、更迅速
$ pip install paddlehub==1.6.1 -i https://pypi.tuna.tsinghua.edu.cn/simple
#升级到最新版本，使用清华源更稳定、更迅速
$ pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple
```
### 人像扣图
```python
# 下载待测试图片
$ wget https://paddlehub.bj.bcebos.com/resources/test_image.jpg
# 通过命令行方式实现人像扣图任务
$ hub run deeplabv3p_xception65_humanseg --input_path test_image.jpg --visualization=True --output_dir="humanseg_output"
```
--2020-07-22 12:19:52-- https://paddlehub.bj.bcebos.com/resources/test_image.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 967120 (944K) [image/jpeg]
Saving to: ‘test_image.jpg’
test_image.jpg 100%[===================>] 944.45K 6.13MB/s in 0.2s
2020-07-22 12:19:53 (6.13 MB/s) - ‘test_image.jpg’ saved [967120/967120]
[{'save_path': 'humanseg_output/test_image.png', 'data': array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]], dtype=float32)}]
![png](../imgs/humanseg_test_res.png)
### 中文分词
```python
#通过命令行方式实现文本分词任务
$ hub run lac --input_text "今天是个好日子"
```
Install Module lac
Downloading lac
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpjcskpj8x/lac
[==================================================] 100.00%
Successfully installed lac-2.1.1
[{'word': ['今天', '是', '个', '好日子'], 'tag': ['TIME', 'v', 'q', 'n']}]
上面的命令中包含四个部分，分别是：
- hub 表示PaddleHub的命令。
- run 调用run执行模型的预测。
- deeplabv3p_xception65_humanseg、lac 表示要调用的算法模型。
- --input_path/--input_text 表示模型的输入数据，图像和文本的输入方式不同。
另外，命令行中`visualization=True`表示将结果可视化输出，`output_dir="humanseg_output"`表示预测结果的保存目录，可以到该路径下查看输出的图片。
再看一个文字识别和一个口罩检测的例子。
### OCR文字识别
```python
# 下载待测试的图片
$ wget https://paddlehub.bj.bcebos.com/model/image/ocr/test_ocr.jpg
# 该Module依赖于第三方库shapely和pyclipper，需提前安装
$ pip install shapely
$ pip install pyclipper
# 通过命令行方式实现文字识别任务
$ hub run chinese_ocr_db_crnn_mobile --input_path test_ocr.jpg --visualization=True --output_dir='ocr_result'
```
--2020-07-22 15:00:50-- https://paddlehub.bj.bcebos.com/model/image/ocr/test_ocr.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 48680 (48K) [image/jpeg]
Saving to: ‘test_ocr.jpg’
test_ocr.jpg 100%[===================>] 47.54K --.-KB/s in 0.02s
2020-07-22 15:00:51 (2.88 MB/s) - ‘test_ocr.jpg’ saved [48680/48680]
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: shapely in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.7.0)
Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/
Requirement already satisfied: pyclipper in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (1.2.0)
[{'save_path': 'ocr_result/ndarray_1595401261.294494.jpg', 'data': [{'text': '纯臻营养护发素', 'confidence': 0.9438689351081848, 'text_box_position': [[24, 36], [304, 34], [304, 72], [24, 74]]}, {'text': '产品信息/参数', 'confidence': 0.9843138456344604, 'text_box_position': [[24, 80], [172, 80], [172, 104], [24, 104]]}, {'text': '（45元/每公斤，100公斤起订）', 'confidence': 0.9210420250892639, 'text_box_position': [[24, 109], [333, 109], [333, 136], [24, 136]]}, {'text': '每瓶22元，1000瓶起订）', 'confidence': 0.9685984253883362, 'text_box_position': [[22, 139], [283, 139], [283, 166], [22, 166]]}, {'text': '【品牌】', 'confidence': 0.9527574181556702, 'text_box_position': [[22, 174], [85, 174], [85, 198], [22, 198]]}, {'text': '：代加工方式/OEMODM', 'confidence': 0.9442129135131836, 'text_box_position': [[90, 176], [301, 176], [301, 196], [90, 196]]}, {'text': '【品名】', 'confidence': 0.8793742060661316, 'text_box_position': [[23, 205], [85, 205], [85, 229], [23, 229]]}, {'text': '：纯臻营养护发素', 'confidence': 0.9230973124504089, 'text_box_position': [[95, 204], [235, 206], [235, 229], [95, 227]]}, {'text': '【产品编号】', 'confidence': 0.9311650395393372, 'text_box_position': [[24, 238], [120, 238], [120, 260], [24, 260]]}, {'text': 'J：YM-X-3011', 'confidence': 0.8866629004478455, 'text_box_position': [[110, 239], [239, 239], [239, 256], [110, 256]]}, {'text': 'ODMOEM', 'confidence': 0.9916308522224426, 'text_box_position': [[414, 233], [430, 233], [430, 304], [414, 304]]}, {'text': '【净含量】：220ml', 'confidence': 0.8709315657615662, 'text_box_position': [[23, 268], [181, 268], [181, 292], [23, 292]]}, {'text': '【适用人群】', 'confidence': 0.9589888453483582, 'text_box_position': [[24, 301], [118, 301], [118, 321], [24, 321]]}, {'text': '：适合所有肤质', 'confidence': 0.935418963432312, 'text_box_position': [[131, 300], [254, 300], [254, 323], [131, 323]]}, {'text': '【主要成分】', 'confidence': 0.9366627335548401, 'text_box_position': [[24, 332], [117, 332], [117, 353], [24, 353]]}, {'text': '鲸蜡硬脂醇', 'confidence': 0.9033458828926086, 'text_box_position': [[138, 331], [235, 331], [235, 351], [138, 351]]}, {'text': '燕麦B-葡聚', 'confidence': 0.8497812747955322, 'text_box_position': [[248, 332], [345, 332], [345, 352], [248, 352]]}, {'text': '椰油酰胺丙基甜菜碱、', 'confidence': 0.8935506939888, 'text_box_position': [[54, 363], [232, 363], [232, 383], [54, 383]]}, {'text': '糖、', 'confidence': 0.8750994205474854, 'text_box_position': [[25, 364], [62, 364], [62, 383], [25, 383]]}, {'text': '泛酯', 'confidence': 0.5581164956092834, 'text_box_position': [[244, 363], [281, 363], [281, 382], [244, 382]]}, {'text': '（成品包材）', 'confidence': 0.9566792845726013, 'text_box_position': [[368, 367], [475, 367], [475, 388], [368, 388]]}, {'text': '【主要功能】', 'confidence': 0.9493741393089294, 'text_box_position': [[24, 395], [119, 395], [119, 416], [24, 416]]}, {'text': '：可紧致头发磷层', 'confidence': 0.9692543745040894, 'text_box_position': [[128, 397], [273, 397], [273, 414], [128, 414]]}, {'text': '美，从而达到', 'confidence': 0.8662520051002502, 'text_box_position': [[265, 395], [361, 395], [361, 415], [265, 415]]}, {'text': '即时持久改善头发光泽的效果，给干燥的头', 'confidence': 0.9690631031990051, 'text_box_position': [[25, 425], [372, 425], [372, 448], [25, 448]]}, {'text': '发足够的滋养', 'confidence': 0.8946213126182556, 'text_box_position': [[26, 457], [136, 457], [136, 477], [26, 477]]}]}]
```python
# 查看预测结果
```
![png](../imgs/ocr_res.jpg)
### 口罩检测
```python
# 下载待测试的图片
$ wget https://paddlehub.bj.bcebos.com/resources/test_mask_detection.jpg
# 通过命令行方式实现文字识别任务
$ hub run pyramidbox_lite_mobile_mask --input_path test_mask_detection.jpg --visualization=True --output_dir='detection_result'
```
--2020-07-22 15:08:11-- https://paddlehub.bj.bcebos.com/resources/test_mask_detection.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.229, 182.61.200.195
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.229|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 299133 (292K) [image/jpeg]
Saving to: ‘test_mask_detection.jpg’
test_mask_detection 100%[===================>] 292.12K --.-KB/s in 0.06s
2020-07-22 15:08:11 (4.55 MB/s) - ‘test_mask_detection.jpg’ saved [299133/299133]
Install Module pyramidbox_lite_mobile_mask
Downloading pyramidbox_lite_mobile_mask
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmp8oes9jid/pyramidbox_lite_mobile_mask
[==================================================] 100.00%
Successfully installed pyramidbox_lite_mobile_mask-1.3.0
Downloading pyramidbox_lite_mobile
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpvhjhlr10/pyramidbox_lite_mobile
[==================================================] 100.00%
[{'data': [{'label': 'MASK', 'confidence': 0.9992434978485107, 'top': 181, 'bottom': 440, 'left': 457, 'right': 654}, {'label': 'MASK', 'confidence': 0.9224318265914917, 'top': 340, 'bottom': 578, 'left': 945, 'right': 1125}, {'label': 'NO MASK', 'confidence': 0.9996706247329712, 'top': 292, 'bottom': 500, 'left': 1166, 'right': 1323}], 'path': 'test_mask_detection.jpg'}]
```python
# 查看预测结果
```
![png](../imgs/test_mask_detection_result.jpg)
### PaddleHub命令行工具简介
PaddleHub的命令行工具在开发时借鉴了Anaconda和PIP等软件包管理的理念，可以方便快捷的完成模型的搜索、下载、安装、升级、预测等功能。 下面概要介绍一下PaddleHub支持的12个命令，详细介绍可查看[命令行参考](http://)章节。：
* install：用于将Module安装到本地，默认安装在{HUB_HOME}/.paddlehub/modules目录下；
* uninstall：卸载本地Module；
* show：用于查看本地已安装Module的属性或者指定目录下确定的Module的属性，包括其名字、版本、描述、作者等信息；
* download：用于下载百度飞桨PaddleHub提供的Module；
* search：通过关键字在服务端检索匹配的Module，当想要查找某个特定模型的Module时，使用search命令可以快速得到结果，例如hub search ssd命令，会查找所有包含了ssd字样的Module，命令支持正则表达式，例如hub search ^s.\*搜索所有以s开头的资源；
* list：列出本地已经安装的Module；
* run：用于执行Module的预测；
* version：显示PaddleHub版本信息；
* help：显示帮助信息；
* clear：PaddleHub在使用过程中会产生一些缓存数据，这部分数据默认存放在${HUB_HOME}/.paddlehub/cache目录下，用户可以通过clear命令来清空缓存；
* autofinetune：用于自动调整Fine-tune任务的超参数，具体使用详情参考[PaddleHub AutoDL Finetuner](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.5/docs/tutorial/autofinetune.md)使用教程；
* config：用于查看和设置Paddlehub相关设置，包括对server地址、日志级别的设置；
* serving：用于一键部署Module预测服务，详细用法见[PaddleHub Serving一键服务部署](https://github.com/PaddlePaddle/PaddleHub/blob/release/v1.5/docs/tutorial/serving.md)。
## 小结
PaddleHub的产品理念是模型即软件，通过Python API或命令行实现模型调用，可快速体验或集成飞桨特色预训练模型。
此外，当用户想用少量数据来优化预训练模型时，PaddleHub也支持迁移学习，通过Fine-tune API，内置多种优化策略，只需少量代码即可完成预训练模型的Fine-tuning。具体可通过后面迁移学习的章节了解。
>值得注意的是，不是所有的Module都支持通过命令行预测 (例如BERT/ERNIE Transformer类模型，一般需要搭配任务进行Fine-tune)， 也不是所有的Module都可用于Fine-tune（例如一般不建议用户使用词法分析LAC模型Fine-tune）。建议提前阅读[预训练模型的介绍文档](https://www.paddlepaddle.org.cn/hublist)了解使用场景。
docs/quick_experience/more_demos.md 已删除 100644 → 0
浏览文件 @ e6b082e1
# PaddleHub更多体验Demos
## PaddleHub官方Demo全集
官方Demo在AI Studio中均可在线运行，类似Notebook的环境，只需通过浏览器即可访问，无需准备环境，非常方便开发者快速体验。
并将持续更新，建议收藏。
[https://aistudio.baidu.com/aistudio/personalcenter/thirdview/79927](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/79927)
## PaddleHub开发者趣味实践作品
以下为前期PaddleHub课程或活动中，开发者们基于PaddleHub创作的趣味实践作品，均收录在AI Studio中，可在线运行，欢迎访问，希望对您有所启发。
1. [布剪刀石头【人脸识别切换本地窗口】](https://aistudio.baidu.com/aistudio/projectdetail/507630)
1. [秋水中的鱼【yesok舞蹈背景抠图转换并动漫风格迁移】](http://aistudio.baidu.com/aistudio/projectdetail/517066)
1. [Ninetailskim【在人脸上玩复古windows弹球】](https://aistudio.baidu.com/aistudio/projectdetail/518861)
1. [乌拉__【监控口罩，语音提醒，后台记录】](https://aistudio.baidu.com/aistudio/projectdetail/506931)
1. [九品炼丹师【影流之绿蛙蔡徐坤，人脸识别加头饰+ 人像分割变分身】](https://aistudio.baidu.com/aistudio/projectdetail/505168)
1. [七年期限【风格迁移以及本地部署】](https://aistudio.baidu.com/aistudio/projectdetail/520453)
1. [Fanas无敌【口红试色项目】](https://aistudio.baidu.com/aistudio/projectdetail/516520)
1. [skywalk163【用paddlehub统计飞桨源代码词频以及词云与人像展示】](https://aistudio.baidu.com/aistudio/projectdetail/519841)
1. [AIStudio261428【人脸识别+漫画表情包】](https://aistudio.baidu.com/aistudio/projectdetail/519616)
1. [土豆芽【六一儿童节邀请卡通人物来做客】](https://aistudio.baidu.com/aistudio/projectdetail/520925)
1. [大熊猫的感觉【变化的口罩】](https://aistudio.baidu.com/aistudio/projectdetail/520996)
1. [kly1997【一键旅游+戴墨镜】](https://aistudio.baidu.com/aistudio/projectdetail/518117)
1. [寞寞_默默【穿越到油画中】](https://aistudio.baidu.com/aistudio/projectdetail/516332)
1. [isse7【创意项目：风格“鬼脸”变换】](https://aistudio.baidu.com/aistudio/projectdetail/515307)
1. [Pda【人脸趣味变】](https://aistudio.baidu.com/aistudio/projectdetail/516306)
1. [Kgkzhiwen【我的新衣】](https://aistudio.baidu.com/aistudio/projectdetail/516663)
1. [哎呀呀好好学习【脸型自动调整】](https://aistudio.baidu.com/aistudio/projectdetail/513640)
1. [Tfboy【证件照换底】](https://aistudio.baidu.com/aistudio/projectdetail/509443)
1. [Leigangblog【我是明星脸】](https://aistudio.baidu.com/aistudio/projectdetail/505537)
1. [wpb3dm【时装模特换装】](https://aistudio.baidu.com/aistudio/projectdetail/519349)
1. [lsvine_bai【女友秒变神秘金发女神】](https://aistudio.baidu.com/aistudio/projectdetail/521784)
1. [Lemonadeqk【简单追星】](https://aistudio.baidu.com/aistudio/projectdetail/520488)
1. [XM1436gr【利用PaddleHub关键点检测实现AI换卡通脸】](https://aistudio.baidu.com/aistudio/projectdetail/514547)
1. [旺仔【人人都是圆眼萌仔】](https://aistudio.baidu.com/aistudio/projectdetail/519222)
1. [Arrowarcher【AI一键换发】](https://aistudio.baidu.com/aistudio/projectdetail/508270)
1. [WHY197598【移物换景基础】](https://aistudio.baidu.com/aistudio/projectdetail/517961)
1. [署名景逸【基于paddlehub人脸关键点检测的疲劳检测】](https://aistudio.baidu.com/aistudio/projectdetail/506024)
1. [thunder95【PaddleHub目光表情投票】](https://aistudio.baidu.com/aistudio/projectdetail/514205)
1. [上弦月C 【坟头蹦迪毕业照】](https://aistudio.baidu.com/aistudio/projectdetail/511253)
1. [如意_鸡蛋【左看像周润发，右看像刘德华】](https://aistudio.baidu.com/aistudio/projectdetail/507231)
docs/quick_experience/python_use_hub.md 已删除 100644 → 0
浏览文件 @ e6b082e1
# 通过Python代码调用方式使用PaddleHub
本页面的代码/命令可在[AIStudio](https://aistudio.baidu.com/aistudio/projectdetail/635335)上在线运行，类似notebook的环境，只需通过浏览器即可访问，无需准备环境，非常方便开发者快速体验。
## 计算机视觉任务的PaddleHub示例
先以计算机视觉任务为例，我们选用一张测试图片test.jpg，分别实现如下四项功能：
* 人像扣图（[deeplabv3p_xception65_humanseg](https://www.paddlepaddle.org.cn/hubdetail?name=deeplabv3p_xception65_humanseg&en_category=ImageSegmentation)）
* 人体部位分割（[ace2p](https://www.paddlepaddle.org.cn/hubdetail?name=ace2p&en_category=ImageSegmentation)）
* 人脸检测（[ultra_light_fast_generic_face_detector_1mb_640](https://www.paddlepaddle.org.cn/hubdetail?name=ultra_light_fast_generic_face_detector_1mb_640&en_category=FaceDetection)）
* 关键点检测（[human_pose_estimation_resnet50_mpii](https://www.paddlepaddle.org.cn/hubdetail?name=human_pose_estimation_resnet50_mpii&en_category=KeyPointDetection)）
>注：如果需要查找PaddleHub中可以调用哪些预训练模型，获取模型名称（如deeplabv3p_xception65_humanseg，后续代码中通过该名称调用模型），请参考[官网文档](https://www.paddlepaddle.org.cn/hublist)，文档中已按照模型类别分好类，方便查找，并且提供了详细的模型介绍。
### 体验前请提前安装好PaddleHub
```python
# 下载安装paddlehub到最新版本，仅第一次运行项目时执行此命令
$ pip install paddlehub==1.6.1 -i https://pypi.tuna.tsinghua.edu.cn/simple #指定版本安装PaddleHub，使用清华源更稳定、更迅速
$ pip install paddlehub --upgrade -i https://pypi.tuna.tsinghua.edu.cn/simple #升级到最新版本，使用清华源更稳定、更迅速
```
### 原图展示
```python
# 下载待测试图片
$ wget https://paddlehub.bj.bcebos.com/resources/test_image.jpg
```
--2020-07-22 12:22:19-- https://paddlehub.bj.bcebos.com/resources/test_image.jpg
Resolving paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)... 182.61.200.195, 182.61.200.229
Connecting to paddlehub.bj.bcebos.com (paddlehub.bj.bcebos.com)|182.61.200.195|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 967120 (944K) [image/jpeg]
Saving to: ‘test_image.jpg.1’
test_image.jpg.1 100%[===================>] 944.45K 5.51MB/s in 0.2s
2020-07-22 12:22:19 (5.51 MB/s) - ‘test_image.jpg.1’ saved [967120/967120]
![png](../imgs/humanseg_test.png)
### 人像扣图
PaddleHub采用模型即软件的设计理念，所有的预训练模型与Python软件包类似，具备版本的概念，通过`hub install`、`hub uninstall`命令可以便捷地完成模型的安装、升级和卸载。
> 使用如下命令默认下载最新版本的模型，如果需要指定版本，可在后面接版本号，如`==1.1.1`。
```python
#安装预训练模型，deeplabv3p_xception65_humanseg是模型名称
$ hub install deeplabv3p_xception65_humanseg
```
Downloading deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpo32jeve0/deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Successfully installed deeplabv3p_xception65_humanseg-1.1.1
```python
# 导入paddlehub库
import paddlehub as hub
# 指定模型名称、待预测的图片路径、输出结果的路径，执行并输出预测结果
module = hub.Module(name="deeplabv3p_xception65_humanseg")
res = module.segmentation(paths = ["./test_image.jpg"], visualization=True, output_dir='humanseg_output')
```
[32m[2020-07-22 12:22:49,474] [ INFO] - Installing deeplabv3p_xception65_humanseg module [0m
Downloading deeplabv3p_xception65_humanseg
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpzrrl1duq/deeplabv3p_xception65_humanseg
[==================================================] 100.00%
[32m[2020-07-22 12:23:11,811] [ INFO] - Successfully installed deeplabv3p_xception65_humanseg-1.1.1 [0m
![png](../imgs/output_8_3.png)
可以看到，使用Python代码调用PaddleHub只需要三行代码即可实现：
```
import paddlehub as hub # 导入PaddleHub代码库
module = hub.Module(name="deeplabv3p_xception65_humanseg") # 指定模型名称
res = module.segmentation(paths = ["./test.jpg"], visualization=True, output_dir='humanseg_output') # 指定模型的输入和输出路径，执行并输出预测结果，其中visualization=True表示将结果可视化输出
```
* 模型名称均通过`hub.Module` API来指定；
* `module.segmentation`用于执行图像分割类的预测任务，不同类型任务设计了不同的预测API，比如人脸检测任务采用`face_detection`函数执行预测，建议调用预训练模型之前先仔细查阅对应的模型介绍文档。
* 预测结果保存在`output_dir='humanseg_output'`目录下，可以到该路径下查看输出的图片。
其他任务的实现方式，均可参考这个“套路”。看一下接下来几个任务如何实现。
### 人体部位分割
```python
#安装预训练模型
$ hub install ace2p
```
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Downloading ace2p
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpfsovt3f8/ace2p
[==================================================] 100.00%
Successfully installed ace2p-1.1.0
```python
# 导入paddlehub库
import paddlehub as hub
# 指定模型名称、待预测的图片路径、输出结果的路径，执行并输出预测结果
module = hub.Module(name="ace2p")
res = module.segmentation(paths = ["./test_image.jpg"], visualization=True, output_dir='ace2p_output')
```
[32m[2020-07-22 12:23:58,027] [ INFO] - Installing ace2p module [0m
Downloading ace2p
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmptrogpj6j/ace2p
[==================================================] 100.00%
[32m[2020-07-22 12:24:22,575] [ INFO] - Successfully installed ace2p-1.1.0 [0m
![png](../imgs/output_12_3.png)
### 人脸检测
```python
#安装预训练模型
$ hub install ultra_light_fast_generic_face_detector_1mb_640
```
Downloading ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpz82xnmy6/ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Successfully installed ultra_light_fast_generic_face_detector_1mb_640-1.1.2
```python
# 导入paddlehub库
import paddlehub as hub
# 指定模型名称、待预测的图片路径、输出结果的路径，执行并输出预测结果
module = hub.Module(name="ultra_light_fast_generic_face_detector_1mb_640")
res = module.face_detection(paths = ["./test_image.jpg"], visualization=True, output_dir='face_detection_output')
```
[32m[2020-07-22 12:25:12,948] [ INFO] - Installing ultra_light_fast_generic_face_detector_1mb_640 module [0m
Downloading ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpw44mo56p/ultra_light_fast_generic_face_detector_1mb_640
[==================================================] 100.00%
[32m[2020-07-22 12:25:14,698] [ INFO] - Successfully installed ultra_light_fast_generic_face_detector_1mb_640-1.1.2
![png](../imgs/output_15_3.png)
### 关键点检测
```python
#安装预训练模型
$ hub install human_pose_estimation_resnet50_mpii
```
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Downloading human_pose_estimation_resnet50_mpii
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmpn_ppwkzq/human_pose_estimation_resnet50_mpii
[======== ] 17.99%
```python
# 导入paddlehub库
import paddlehub as hub
# 指定模型名称、待预测的图片路径、输出结果的路径，执行并输出预测结果
module = hub.Module(name="human_pose_estimation_resnet50_mpii")
res = module.keypoint_detection(paths = ["./test_image.jpg"], visualization=True, output_dir='keypoint_output')
```
[32m[2020-07-23 11:27:33,989] [ INFO] - Installing human_pose_estimation_resnet50_mpii module [0m
[32m[2020-07-23 11:27:33,992] [ INFO] - Module human_pose_estimation_resnet50_mpii already installed in /home/aistudio/.paddlehub/modules/human_pose_estimation_resnet50_mpii [0m
image saved in keypoint_output/test_imagetime=1595474855.jpg
![png](../imgs/output_18_2.png)
## 自然语言处理任务的PaddleHub示例
再看两个自然语言处理任务的示例，下面以中文分词和情感分类的任务为例介绍。
* 中文分词（[lac](https://www.paddlepaddle.org.cn/hubdetail?name=lac&en_category=LexicalAnalysis)）
* 情感分析（[senta_bilstm](https://www.paddlepaddle.org.cn/hubdetail?name=senta_bilstm&en_category=SentimentAnalysis)）
### 中文分词
```python
#安装预训练模型
$ hub install lac
```
2020-07-22 10:03:09,866-INFO: font search path ['/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/ttf', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/afm', '/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/matplotlib/mpl-data/fonts/pdfcorefonts']
2020-07-22 10:03:10,208-INFO: generated new fontManager
Downloading lac
[==================================================] 100.00%
Uncompress /home/aistudio/.paddlehub/tmp/tmp8ukaz690/lac
[==================================================] 100.00%
Successfully installed lac-2.1.1
```python
# 导入paddlehub库
import paddlehub as hub
# 指定模型名称、待分词的文本，执行并输出预测结果
lac = hub.Module(name="lac")
test_text = ["1996年，曾经是微软员工的加布·纽维尔和麦克·哈灵顿一同创建了Valve软件公司。他们在1996年下半年从id software取得了雷神之锤引擎的使用许可，用来开发半条命系列。"]
res = lac.lexical_analysis(texts = test_text)
# 打印预测结果
print("中文词法分析结果：", res)
```
[32m[2020-07-22 10:03:18,439] [ INFO] - Installing lac module[0m
[32m[2020-07-22 10:03:18,531] [ INFO] - Module lac already installed in /home/aistudio/.paddlehub/modules/lac [0m
中文词法分析结果： [{'word': ['1996年', '，', '曾经', '是', '微软', '员工', '的', '加布·纽维尔', '和', '麦克·哈灵顿', '一同', '创建', '了', 'Valve软件公司', '。', '他们', '在', '1996年下半年', '从', 'id', ' ', 'software', '取得', '了', '雷神之锤', '引擎', '的', '使用', '许可', '，', '用来', '开发', '半条命', '系列', '。'], 'tag': ['TIME', 'w', 'd', 'v', 'ORG', 'n', 'u', 'PER', 'c', 'PER', 'd', 'v', 'u', 'ORG', 'w', 'r', 'p', 'TIME', 'p', 'nz', 'w', 'n', 'v', 'u', 'n', 'n', 'u', 'vn', 'vn', 'w', 'v', 'v', 'n', 'n', 'w']}]
可以看到，与计算机视觉任务相比，输入和输出接口（这里需要输入文本，以函数参数的形式传入）存在差异，这与任务类型相关，具体可查看对应预训练模型的API介绍。
### 情感分类
```python
#安装预训练模型
$ hub install senta_bilstm
```
Module senta_bilstm-1.1.0 already installed in /home/aistudio/.paddlehub/modules/senta_bilstm
```python
import paddlehub as hub
senta = hub.Module(name="senta_bilstm")
test_text = ["味道不错，确实不算太辣，适合不能吃辣的人。就在长江边上，抬头就能看到长江的风景。鸭肠、黄鳝都比较新鲜。"]
res = senta.sentiment_classify(texts = test_text)
print("情感分析结果：", res)
```
[32m[2020-07-22 10:34:06,922] [ INFO] - Installing senta_bilstm module [0m
[32m[2020-07-22 10:34:06,984] [ INFO] - Module senta_bilstm already installed in /home/aistudio/.paddlehub/modules/senta_bilstm
[32m[2020-07-22 10:34:08,937] [ INFO] - Installing lac module[0m
[32m[2020-07-22 10:34:08,939] [ INFO] - Module lac already installed in /home/aistudio/.paddlehub/modules/lac [0m
情感分析结果： [{'text': '味道不错，确实不算太辣，适合不能吃辣的人。就在长江边上，抬头就能看到长江的风景。鸭肠、黄鳝都比较新鲜。', 'sentiment_label': 1, 'sentiment_key': 'positive', 'positive_probs': 0.9771, 'negative_probs': 0.0229}]
## 总结
PaddleHub提供了丰富的预训练模型，包括图像分类、语义模型、视频分类、图像生成、图像分割、文本审核、关键点检测等主流模型，只需要3行Python代码即可快速调用，即时输出预测结果，非常方便。您可以尝试一下，从[预训练模型列表](https://www.paddlepaddle.org.cn/hublist)中选择一些模型体验一下。
docs/quick_experience/quick_index.rst 已删除 100644 → 0
浏览文件 @ e6b082e1
快速体验
==================
安装PaddleHub成功后，可以通过如下几个章节快速体验PaddleHub的推理功能。
.. toctree::
:maxdepth: 1
通过命令行调用方式使用PaddleHub<cmd_quick_run>
通过Python代码调用方式使用PaddleHub<python_use_hub>
PaddleHub更多体验Demos<more_demos>
docs/tutorial/how_to_finetune.md 已删除 100644 → 0
浏览文件 @ e6b082e1
此差异已折叠。
docs/tutorial/tutorial_index.rst 已删除 100644 → 0
浏览文件 @ e6b082e1
教程
==================
以下是关于PaddleHub的使用教程，介绍了命令行使用、如何自定义数据完成Finetune、如何自定义迁移任务、如何服务化部署预训练模型、如何获取ERNIE/BERT Embedding、如何用word2vec完成语义相似度计算、ULMFit优化策略介绍、如何使用超参优化AutoDL Finetuner、如何用Hook机制改写Task内置方法。
详细信息，参考以下教程：
.. toctree::
:maxdepth: 1
命令行工具<cmdintro>
自定义数据<how_to_load_data>
Fine-tune模型转化为PaddleHub Module<finetuned_model_to_module.md>
迁移学习<how_to_finetune>
服务化部署<serving>
文本Embedding服务<bert_service>
自定义任务<how_to_define_task>
语义相似度计算<sentence_sim>
ULMFit优化策略<strategy_exp>
超参优化<autofinetune>
Hook机制<hook>
hub_module/scripts/configs/dcscn.yml 已删除 100644 → 0
浏览文件 @ e6b082e1
name: dcscn
dir: "modules/image/super_resolution/dcscn"
exclude:
- README.md
resources:
-
url: https://bj.bcebos.com/paddlehub/model/image/image_editing/dcscn_model.tar.gz
dest: .
uncompress: True
hub_module/scripts/configs/falsr_b.yml 已删除 100644 → 0
浏览文件 @ e6b082e1
name: falsr_b
dir: "modules/image/super_resolution/falsr_b"
exclude:
- README.md
resources:
-
url: https://bj.bcebos.com/paddlehub/model/image/image_editing/falsr_B_model.tar.gz
dest: .
uncompress: True
hub_module/scripts/configs/humanseg_lite.yml 已删除 100644 → 0
浏览文件 @ e6b082e1
name: humanseg_lite
dir: "modules/image/semantic_segmentation/humanseg_lite"
exclude:
- README.md
resources:
-
url: https://bj.bcebos.com/paddlehub/model/image/semantic_segmentation/humanseg_lite_inference.tar.gz
dest: .
uncompress: True
hub_module/scripts/configs/humanseg_mobile.yml 已删除 100644 → 0
浏览文件 @ e6b082e1
name: humanseg_mobile
dir: "modules/image/semantic_segmentation/humanseg_mobile"
exclude:
- README.md
resources:
-
url: https://bj.bcebos.com/paddlehub/model/image/semantic_segmentation/humanseg_mobile_inference.tar.gz
dest: .
uncompress: True
hub_module/tests/image_dataset/human_segmentation/path.txt 已删除 100644 → 0
浏览文件 @ e6b082e1
../image_dataset/human_segmentation/image/ache-adult-depression-expression-41253.jpg
../image_dataset/human_segmentation/image/allergy-cold-disease-flu-41284.jpg
../image_dataset/human_segmentation/image/bored-female-girl-people-41321.jpg
../image_dataset/human_segmentation/image/colors-hairdresser-cutting-colorimetry-159780.jpg
../image_dataset/human_segmentation/image/pexels-photo-206339.jpg
hub_module/tests/unittests/test_dcscn.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
imgpath = [
'../image_dataset/super_resolution/BSD100_001.png',
'../image_dataset/super_resolution/BSD100_002.png',
'../image_dataset/super_resolution/BSD100_003.png',
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.sr_model = hub.Module(name="dcscn")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.sr_model = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
img = cv2.imread(imgpath[0])
result = self.sr_model.super_resolution(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
for pic_path in imgpath:
img = cv2.imread(pic_path)
result = self.sr_model.super_resolution(
images=[img],
output_dir='test_dcscn_model_output',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.sr_model.save_inference_model(
dirname='test_dcscn_model', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
hub_module/tests/unittests/test_falsr_A.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
imgpath = [
'../image_dataset/super_resolution/BSD100_001.png',
'../image_dataset/super_resolution/BSD100_002.png',
'../image_dataset/super_resolution/BSD100_003.png',
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.sr_model = hub.Module(name="falsr_A")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.sr_model = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
img = cv2.imread(imgpath[0])
result = self.sr_model.super_resolution(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
for pic_path in imgpath:
img = cv2.imread(pic_path)
result = self.sr_model.super_resolution(
images=[img],
output_dir='test_falsr_A_model_output',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.sr_model.save_inference_model(
dirname='test_falsr_A_model', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
hub_module/tests/unittests/test_falsr_B.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
imgpath = [
'../image_dataset/super_resolution/BSD100_001.png',
'../image_dataset/super_resolution/BSD100_002.png',
'../image_dataset/super_resolution/BSD100_003.png',
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.sr_model = hub.Module(name="falsr_B")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.sr_model = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
img = cv2.imread(imgpath[0])
result = self.sr_model.super_resolution(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
for pic_path in imgpath:
img = cv2.imread(pic_path)
result = self.sr_model.super_resolution(
images=[img],
output_dir='test_falsr_B_model_output',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.sr_model.save_inference_model(
dirname='test_falsr_B_model', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
hub_module/tests/unittests/test_falsr_C.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
imgpath = [
'../image_dataset/super_resolution/BSD100_001.png',
'../image_dataset/super_resolution/BSD100_002.png',
'../image_dataset/super_resolution/BSD100_003.png',
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.sr_model = hub.Module(name="falsr_C")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.sr_model = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
img = cv2.imread(imgpath[0])
result = self.sr_model.super_resolution(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
for pic_path in imgpath:
img = cv2.imread(pic_path)
result = self.sr_model.super_resolution(
images=[img],
output_dir='test_falsr_C_model_output',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.sr_model.save_inference_model(
dirname='test_falsr_C_model', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
hub_module/tests/unittests/test_humanseg_lite.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
pic_dir = '../image_dataset/human_segmentation/image/'
imgpath = [
'../image_dataset/human_segmentation/image/ache-adult-depression-expression-41253.jpg',
'../image_dataset/human_segmentation/image/allergy-cold-disease-flu-41284.jpg',
'../image_dataset/human_segmentation/image/bored-female-girl-people-41321.jpg',
'../image_dataset/human_segmentation/image/colors-hairdresser-cutting-colorimetry-159780.jpg',
'../image_dataset/human_segmentation/image/pexels-photo-206339.jpg'
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.human_seg = hub.Module(name="humanseg_lite")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.human_seg = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
pics_path_list = [
os.path.join(pic_dir, f) for f in os.listdir(pic_dir)
]
img = cv2.imread(pics_path_list[0])
result = self.human_seg.segment(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_batch(self):
with fluid.program_guard(self.test_prog):
pics_path_list = [
os.path.join(pic_dir, f) for f in os.listdir(pic_dir)
]
result = self.human_seg.segment(
paths=imgpath,
batch_size=2,
output_dir='batch_output_shuffle',
use_gpu=False,
visualization=True)
print(result)
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
pics_path_list = [
os.path.join(pic_dir, f) for f in os.listdir(pic_dir)
]
pics_ndarray = list()
for pic_path in pics_path_list:
img = cv2.imread(pic_path)
result = self.human_seg.segment(
images=[img],
output_dir='ndarray_output_shuffle',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.human_seg.save_inference_model(
dirname='humanseg_lite', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_batch'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
hub_module/tests/unittests/test_humanseg_mobile.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import time
import unittest
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
pic_dir = '../image_dataset/human_segmentation/image/'
imgpath = [
'../image_dataset/human_segmentation/image/ache-adult-depression-expression-41253.jpg',
'../image_dataset/human_segmentation/image/allergy-cold-disease-flu-41284.jpg',
'../image_dataset/human_segmentation/image/bored-female-girl-people-41321.jpg',
'../image_dataset/human_segmentation/image/colors-hairdresser-cutting-colorimetry-159780.jpg',
'../image_dataset/human_segmentation/image/pexels-photo-206339.jpg'
]
class TestHumanSeg(unittest.TestCase):
@classmethod
def setUpClass(self):
"""Prepare the environment once before execution of all tests.\n"""
self.human_seg = hub.Module(name="humanseg_mobile")
@classmethod
def tearDownClass(self):
"""clean up the environment after the execution of all tests.\n"""
self.human_seg = None
def setUp(self):
"Call setUp() to prepare environment\n"
self.test_prog = fluid.Program()
def tearDown(self):
"Call tearDown to restore environment.\n"
self.test_prog = None
def test_single_pic(self):
with fluid.program_guard(self.test_prog):
pics_path_list = [
os.path.join(pic_dir, f) for f in os.listdir(pic_dir)
]
img = cv2.imread(pics_path_list[0])
result = self.human_seg.segment(
images=[img], use_gpu=False, visualization=True)
print(result[0]['data'])
def test_batch(self):
with fluid.program_guard(self.test_prog):
result = self.human_seg.segment(
paths=imgpath,
batch_size=2,
output_dir='batch_output_hrnet',
use_gpu=False,
visualization=True)
print(result)
def test_ndarray(self):
with fluid.program_guard(self.test_prog):
pics_path_list = [
os.path.join(pic_dir, f) for f in os.listdir(pic_dir)
]
pics_ndarray = list()
for pic_path in pics_path_list:
result = self.human_seg.segment(
images=[cv2.imread(pic_path)],
output_dir='ndarray_output_hrnet',
use_gpu=False,
visualization=True)
def test_save_inference_model(self):
with fluid.program_guard(self.test_prog):
self.human_seg.save_inference_model(
dirname='humanseg_mobile', combined=True)
if __name__ == "__main__":
suite = unittest.TestSuite()
suite.addTest(TestHumanSeg('test_single_pic'))
suite.addTest(TestHumanSeg('test_batch'))
suite.addTest(TestHumanSeg('test_ndarray'))
suite.addTest(TestHumanSeg('test_save_inference_model'))
runner = unittest.TextTestRunner(verbosity=2)
runner.run(suite)
modules/image/Image_gan/humanseg/humanseg_lite/README.md 已删除 100644 → 0
浏览文件 @ e6b082e1
## 模型概述
HumanSeg_lite是基于ShuffleNetV2网络结构的基础上进行优化的人像分割模型，进一步减小了网络规模，网络大小只有541K，量化后只有187K，适用于手机自拍人像分割等实时分割场景。
## 命令行预测
```
hub run humanseg_lite --input_path "/PATH/TO/IMAGE"
```
## API
```python
def segment(images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_lite_output')
```
预测API，用于人像分割。
**参数**
* images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* paths (list\[str\]): 图片的路径；
* batch\_size (int): batch 的大小；
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* visualization (bool): 是否将识别结果保存为图片文件；
* output\_dir (str): 图片的保存路径。
**返回**
* res (list\[dict\]): 识别结果的列表，列表中每一个元素为 dict，关键字有 'save\_path', 'data'，对应的取值为：
* save\_path (str, optional): 可视化图片的保存路径（仅当visualization=True时存在）；
* data (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-255 (0为全透明，255为不透明)，也即取值越大的像素点越可能为人体，取值越小的像素点越可能为背景。
```python
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
```
预测API，用于逐帧对视频人像分割。
**参数**
* frame_org (numpy.ndarray): 单帧图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* frame_id (int): 当前帧的编号；
* prev_gray (numpy.ndarray): 前一帧输入网络图像的灰度图；
* prev_cfd (numpy.ndarray): 前一帧光流追踪图和预测结果融合图
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
**返回**
* img_matting (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-1 (0为全透明，1为不透明)。
* cur_gray (numpy.ndarray): 当前帧输入网络图像的灰度图；
* optflow_map (numpy.ndarray): 当前帧光流追踪图和预测结果融合图
```python
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_lite_video_result'):
```
预测API，用于视频人像分割。
**参数**
* video\_path (str): 待分割视频路径。若为None，则从本地摄像头获取视频，并弹出窗口显示在线分割结果。
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* save\_dir (str): 视频保存路径，仅在video\_path不为None时启用，保存离线视频处理结果。
```python
def save_inference_model(dirname='humanseg_lite_model',
model_filename=None,
params_filename=None,
combined=True)
```
将模型保存到指定路径。
**参数**
* dirname: 存在模型的目录名称
* model\_filename: 模型文件名称，默认为\_\_model\_\_
* params\_filename: 参数文件名称，默认为\_\_params\_\_(仅当`combined`为True时生效)
* combined: 是否将参数保存到统一的一个文件中
## 代码示例
图片分割及视频分割代码示例：
```python
import cv2
import paddlehub as hub
human_seg = hub.Module('humanseg_lite')
im = cv2.imread('/PATH/TO/IMAGE')
#visualization=True可以用于查看超分图片效果，可设置为False提升运行速度。
res = human_seg.segment(images=[im],visualization=True)
print(res[0]['data'])
human_seg.video_segment('/PATH/TO/VIDEO')
human_seg.save_inference_model('/PATH/TO/SAVE/MODEL')
```
视频流预测代码示例：
```python
import cv2
import numpy as np
import paddlehub as hub
human_seg = hub.Module('humanseg_lite')
cap_video = cv2.VideoCapture('\PATH\TO\VIDEO')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'humanseg_lite_video.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps, (width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = human_seg.video_stream_segment(frame_org=frame_org, frame_id=cap_video.get(1), prev_gray=prev_gray, prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
```
## 服务部署
PaddleHub Serving可以部署一个人像分割的在线服务。
## 第一步：启动PaddleHub Serving
运行启动命令：
```shell
$ hub serving start -m humanseg_lite
```
这样就完成了一个人像分割的服务化API的部署，默认端口号为8866。
**NOTE:** 如使用GPU预测，则需要在启动服务之前，设置CUDA\_VISIBLE\_DEVICES环境变量，否则不用设置。
## 第二步：发送预测请求
配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
org_im = cv2.imread('PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/humanseg_lite"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 保存图片
mask =cv2.cvtColor(base64_to_cv2(r.json()["results"][0]['data']), cv2.COLOR_BGR2GRAY)
rgba = np.concatenate((org_im, np.expand_dims(mask, axis=2)), axis=2)
cv2.imwrite("segment_human_lite.png", rgba)
```
### 查看代码
https://github.com/PaddlePaddle/PaddleSeg/tree/develop/contrib/HumanSeg
### 依赖
paddlepaddle >= 1.8.0
paddlehub >= 1.7.1
modules/image/Image_gan/humanseg/humanseg_lite/data_feed.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
from collections import OrderedDict
import cv2
import numpy as np
from PIL import Image
__all__ = ['reader', 'preprocess_v']
def preprocess_v(img, w, h):
img = cv2.resize(img, (w, h), cv2.INTER_LINEAR).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
return img
def reader(images=None, paths=None):
"""
Preprocess to yield image.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C]
paths (list[str]): paths to images.
Yield:
each (collections.OrderedDict): info of original image, preprocessed image.
"""
component = list()
if paths:
for im_path in paths:
each = OrderedDict()
assert os.path.isfile(
im_path), "The {} isn't a valid file path.".format(im_path)
#print(im_path)
im = cv2.imread(im_path).astype('float32')
each['org_im'] = im
each['org_im_path'] = im_path
each['org_im_shape'] = im.shape
component.append(each)
if images is not None:
assert type(images) is list, "images should be a list."
for im in images:
each = OrderedDict()
each['org_im'] = im
each['org_im_path'] = 'ndarray_time={}'.format(
round(time.time(), 6) * 1e6)
each['org_im_shape'] = im.shape
component.append(each)
for element in component:
img = element['org_im'].copy()
img = cv2.resize(img, (192, 192)).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
element['image'] = img
yield element
modules/image/Image_gan/humanseg/humanseg_lite/module.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import ast
import os
import os.path as osp
import argparse
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
from paddle.fluid.core import PaddleTensor, AnalysisConfig, create_paddle_predictor
from paddlehub.module.module import moduleinfo, runnable, serving
from humanseg_lite.processor import postprocess, base64_to_cv2, cv2_to_base64, check_dir
from humanseg_lite.data_feed import reader, preprocess_v
from humanseg_lite.optimal import postprocess_v, threshold_mask
@moduleinfo(
name="humanseg_lite",
type="CV/semantic_segmentation",
author="paddlepaddle",
author_email="",
summary="humanseg_lite is a semantic segmentation model.",
version="1.1.0")
class ShufflenetHumanSeg(hub.Module):
def _initialize(self):
self.default_pretrained_model_path = os.path.join(
self.directory, "humanseg_lite_inference")
self._set_config()
def _set_config(self):
"""
predictor config setting
"""
self.model_file_path = os.path.join(self.default_pretrained_model_path,
'__model__')
self.params_file_path = os.path.join(self.default_pretrained_model_path,
'__params__')
cpu_config = AnalysisConfig(self.model_file_path, self.params_file_path)
cpu_config.disable_glog_info()
cpu_config.disable_gpu()
self.cpu_predictor = create_paddle_predictor(cpu_config)
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
use_gpu = True
except:
use_gpu = False
if use_gpu:
gpu_config = AnalysisConfig(self.model_file_path,
self.params_file_path)
gpu_config.disable_glog_info()
gpu_config.enable_use_gpu(
memory_pool_init_size_mb=1000, device_id=0)
self.gpu_predictor = create_paddle_predictor(gpu_config)
def segment(self,
images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_lite_output'):
"""
API for human segmentation.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C], the color space is BGR.
paths (list[str]): The paths of images.
batch_size (int): batch size.
use_gpu (bool): Whether to use gpu.
visualization (bool): Whether to save image or not.
output_dir (str): The path to store output images.
Returns:
res (list[dict]): each element in the list is a dict, the keys and values are:
save_path (str, optional): the path to save images. (Exists only if visualization is True)
data (numpy.ndarray): data of post processed image.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly. If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
all_data = list()
for yield_data in reader(images, paths):
all_data.append(yield_data)
total_num = len(all_data)
loop_num = int(np.ceil(total_num / batch_size))
res = list()
for iter_id in range(loop_num):
batch_data = list()
handle_id = iter_id * batch_size
for image_id in range(batch_size):
try:
batch_data.append(all_data[handle_id + image_id])
except:
pass
# feed batch image
batch_image = np.array([data['image'] for data in batch_data])
batch_image = PaddleTensor(batch_image.copy())
output = self.gpu_predictor.run([
batch_image
]) if use_gpu else self.cpu_predictor.run([batch_image])
output = output[1].as_ndarray()
output = np.expand_dims(output[:, 1, :, :], axis=1)
# postprocess one by one
for i in range(len(batch_data)):
out = postprocess(
data_out=output[i],
org_im=batch_data[i]['org_im'],
org_im_shape=batch_data[i]['org_im_shape'],
org_im_path=batch_data[i]['org_im_path'],
output_dir=output_dir,
visualization=visualization)
res.append(out)
return res
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
"""
API for human video segmentation.
Args:
frame_org (numpy.ndarray): frame data, shape of each is [H, W, C], the color space is BGR.
frame_id (int): index of the frame to be decoded.
prev_gray (numpy.ndarray): gray scale image of last frame, shape of each is [H, W]
prev_cfd (numpy.ndarray): fusion image from optical flow image and segment result, shape of each is [H, W]
use_gpu (bool): Whether to use gpu.
Returns:
img_matting (numpy.ndarray): data of segmentation mask.
cur_gray (numpy.ndarray): gray scale image of current frame, shape of each is [H, W]
optflow_map (numpy.ndarray): optical flow image of current frame, shape of each is [H, W]
"""
resize_h = 192
resize_w = 192
is_init = True
width = int(frame_org.shape[0])
height = int(frame_org.shape[1])
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
if frame_id == 1:
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
else:
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (height, width), cv2.INTER_LINEAR)
return [img_matting, cur_gray, optflow_map]
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_lite_video_result'):
"""
API for human video segmentation.
Args:
video_path (str): The path to take the video under preprocess. If video_path is None, it will capture
the vedio from your camera.
use_gpu (bool): Whether to use gpu.
save_dir (str): The path to store output video.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly. "
"If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
resize_h = 192
resize_w = 192
if not video_path:
cap_video = cv2.VideoCapture(0)
else:
cap_video = cv2.VideoCapture(video_path)
if not cap_video.isOpened():
raise IOError("Error opening video stream or file, "
"--video_path whether existing: {}"
" or camera whether working".format(video_path))
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
is_init = True
fps = cap_video.get(cv2.CAP_PROP_FPS)
if video_path is not None:
print('Please wait. It is computing......')
if not osp.exists(save_dir):
os.makedirs(save_dir)
save_path = osp.join(save_dir, 'result' + '.avi')
cap_out = cv2.VideoWriter(
save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
else:
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cv2.imshow('HumanSegmentation', comb)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap_video.release()
def save_inference_model(self,
dirname='humanseg_lite_model',
model_filename=None,
params_filename=None,
combined=True):
if combined:
model_filename = "__model__" if not model_filename else model_filename
params_filename = "__params__" if not params_filename else params_filename
place = fluid.CPUPlace()
exe = fluid.Executor(place)
program, feeded_var_names, target_vars = fluid.io.load_inference_model(
dirname=self.default_pretrained_model_path,
model_filename=model_filename,
params_filename=params_filename,
executor=exe)
fluid.io.save_inference_model(
dirname=dirname,
main_program=program,
executor=exe,
feeded_var_names=feeded_var_names,
target_vars=target_vars,
model_filename=model_filename,
params_filename=params_filename)
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.segment(images=images_decode, **kwargs)
results = [{
'data': cv2_to_base64(result['data'])
} for result in results]
return results
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.segment(
paths=[args.input_path],
batch_size=args.batch_size,
use_gpu=args.use_gpu,
output_dir=args.output_dir,
visualization=args.visualization)
if args.save_dir is not None:
check_dir(args.save_dir)
self.save_inference_model(args.save_dir)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--use_gpu',
type=ast.literal_eval,
default=False,
help="whether use GPU or not")
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='humanseg_lite_output',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='humanseg_lite_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=ast.literal_eval,
default=False,
help="whether to save output as images.")
self.arg_config_group.add_argument(
'--batch_size',
type=ast.literal_eval,
default=1,
help="batch size.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
if __name__ == "__main__":
m = ShufflenetHumanSeg()
#shuffle.video_segment()
img = cv2.imread('photo.jpg')
# res = m.segment(images=[img], visualization=True)
# print(res[0]['data'])
# m.video_segment('')
cap_video = cv2.VideoCapture('video_test.mp4')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'result_frame.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path,
cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = m.video_stream_segment(
frame_org=frame_org,
frame_id=cap_video.get(1),
prev_gray=prev_gray,
prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(
np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
modules/image/Image_gan/humanseg/humanseg_lite/optimal.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*
import numpy as np
def human_seg_tracking(pre_gray, cur_gray, prev_cfd, dl_weights, disflow):
"""计算光流跟踪匹配点和光流图
输入参数:
pre_gray: 上一帧灰度图
cur_gray: 当前帧灰度图
prev_cfd: 上一帧光流图
dl_weights: 融合权重图
disflow: 光流数据结构
返回值:
is_track: 光流点跟踪二值图，即是否具有光流点匹配
track_cfd: 光流跟踪图
"""
check_thres = 8
h, w = pre_gray.shape[:2]
track_cfd = np.zeros_like(prev_cfd)
is_track = np.zeros_like(pre_gray)
flow_fw = disflow.calc(pre_gray, cur_gray, None)
flow_bw = disflow.calc(cur_gray, pre_gray, None)
flow_fw = np.round(flow_fw).astype(np.int)
flow_bw = np.round(flow_bw).astype(np.int)
y_list = np.array(range(h))
x_list = np.array(range(w))
yv, xv = np.meshgrid(y_list, x_list)
yv, xv = yv.T, xv.T
cur_x = xv + flow_fw[:, :, 0]
cur_y = yv + flow_fw[:, :, 1]
# 超出边界不跟踪
not_track = (cur_x < 0) + (cur_x >= w) + (cur_y < 0) + (cur_y >= h)
flow_bw[~not_track] = flow_bw[cur_y[~not_track], cur_x[~not_track]]
not_track += (np.square(flow_fw[:, :, 0] + flow_bw[:, :, 0]) +
np.square(flow_fw[:, :, 1] + flow_bw[:, :, 1])) >= check_thres
track_cfd[cur_y[~not_track], cur_x[~not_track]] = prev_cfd[~not_track]
is_track[cur_y[~not_track], cur_x[~not_track]] = 1
not_flow = np.all(
np.abs(flow_fw) == 0, axis=-1) * np.all(
np.abs(flow_bw) == 0, axis=-1)
dl_weights[cur_y[not_flow], cur_x[not_flow]] = 0.05
return track_cfd, is_track, dl_weights
def human_seg_track_fuse(track_cfd, dl_cfd, dl_weights, is_track):
"""光流追踪图和人像分割结构融合
输入参数:
track_cfd: 光流追踪图
dl_cfd: 当前帧分割结果
dl_weights: 融合权重图
is_track: 光流点匹配二值图
返回
cur_cfd: 光流跟踪图和人像分割结果融合图
"""
fusion_cfd = dl_cfd.copy()
is_track = is_track.astype(np.bool)
fusion_cfd[is_track] = dl_weights[is_track] * dl_cfd[is_track] + (
1 - dl_weights[is_track]) * track_cfd[is_track]
# 确定区域
index_certain = ((dl_cfd > 0.9) + (dl_cfd < 0.1)) * is_track
index_less01 = (dl_weights < 0.1) * index_certain
fusion_cfd[index_less01] = 0.3 * dl_cfd[index_less01] + 0.7 * track_cfd[
index_less01]
index_larger09 = (dl_weights >= 0.1) * index_certain
fusion_cfd[index_larger09] = 0.4 * dl_cfd[index_larger09] + 0.6 * track_cfd[
index_larger09]
return fusion_cfd
def threshold_mask(img, thresh_bg, thresh_fg):
dst = (img / 255.0 - thresh_bg) / (thresh_fg - thresh_bg)
dst[np.where(dst > 1)] = 1
dst[np.where(dst < 0)] = 0
return dst.astype(np.float32)
def postprocess_v(cur_gray, scoremap, prev_gray, pre_cfd, disflow, is_init):
"""光流优化
Args:
cur_gray : 当前帧灰度图
pre_gray : 前一帧灰度图
pre_cfd ：前一帧融合结果
scoremap : 当前帧分割结果
difflow : 光流
is_init : 是否第一帧
Returns:
fusion_cfd : 光流追踪图和预测结果融合图
"""
h, w = scoremap.shape
cur_cfd = scoremap.copy()
if is_init:
if h <= 64 or w <= 64:
disflow.setFinestScale(1)
elif h <= 160 or w <= 160:
disflow.setFinestScale(2)
else:
disflow.setFinestScale(3)
fusion_cfd = cur_cfd
else:
weights = np.ones((h, w), np.float32) * 0.3
track_cfd, is_track, weights = human_seg_tracking(
prev_gray, cur_gray, pre_cfd, weights, disflow)
fusion_cfd = human_seg_track_fuse(track_cfd, cur_cfd, weights, is_track)
return fusion_cfd
modules/image/Image_gan/humanseg/humanseg_lite/processor.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
import base64
import cv2
import numpy as np
__all__ = ['cv2_to_base64', 'base64_to_cv2', 'postprocess']
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
def postprocess(data_out, org_im, org_im_shape, org_im_path, output_dir,
visualization):
"""
Postprocess output of network. one image at a time.
Args:
data_out (numpy.ndarray): output of network.
org_im (numpy.ndarray): original image.
org_im_shape (list): shape pf original image.
org_im_path (list): path of riginal image.
output_dir (str): output directory to store image.
visualization (bool): whether to save image or not.
Returns:
result (dict): The data of processed image.
"""
result = dict()
for logit in data_out:
logit = (logit * 255).astype(np.uint8)
logit = cv2.resize(logit, (org_im_shape[1], org_im_shape[0]))
rgba = np.concatenate((org_im, np.expand_dims(logit, axis=2)), axis=2)
if visualization:
check_dir(output_dir)
save_im_path = get_save_image_name(org_im, org_im_path, output_dir)
cv2.imwrite(save_im_path, rgba)
result['save_path'] = save_im_path
result['data'] = logit
else:
result['data'] = logit
print("result['data'] shape", result['data'].shape)
return result
def check_dir(dir_path):
if not os.path.exists(dir_path):
os.makedirs(dir_path)
elif os.path.isfile(dir_path):
os.remove(dir_path)
os.makedirs(dir_path)
def get_save_image_name(org_im, org_im_path, output_dir):
"""
Get save image name from source image path.
"""
# name prefix of orginal image
org_im_name = os.path.split(org_im_path)[-1]
im_prefix = os.path.splitext(org_im_name)[0]
ext = '.png'
# save image path
save_im_path = os.path.join(output_dir, im_prefix + ext)
if os.path.exists(save_im_path):
save_im_path = os.path.join(
output_dir, im_prefix + 'time={}'.format(int(time.time())) + ext)
return save_im_path
modules/image/Image_gan/humanseg/humanseg_mobile/README.md 已删除 100644 → 0
浏览文件 @ e6b082e1
## 模型概述
HumanSeg-mobile是基于HRNet(Deep High-Resolution Representation Learning for Visual Recognition)的人像分割网络。HRNet在特征提取过程中保持了高分辨率的信息，保持了物体的细节信息，并可通过控制每个分支的通道数调整模型的大小。HumanSeg-mobile采用了HRNet_w18_small_v1的网络结构，模型大小只有5.8M， 适用于移动端或服务端CPU的前置摄像头场景。
## 命令行预测
```
hub run humanseg_mobile --input_path "/PATH/TO/IMAGE"
```
## API
```python
def segment(images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_mobile_output')
```
预测API，用于人像分割。
**参数**
* images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* paths (list\[str\]): 图片的路径；
* batch\_size (int): batch 的大小；
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* visualization (bool): 是否将识别结果保存为图片文件；
* output\_dir (str): 图片的保存路径。
**返回**
* res (list\[dict\]): 识别结果的列表，列表中每一个元素为 dict，关键字有 'save\_path', 'data'，对应的取值为：
* save\_path (str, optional): 可视化图片的保存路径（仅当visualization=True时存在）；
* data (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-255 (0为全透明，255为不透明)，也即取值越大的像素点越可能为人体，取值越小的像素点越可能为背景。
```python
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
```
预测API，用于逐帧对视频人像分割。
**参数**
* frame_org (numpy.ndarray): 单帧图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* frame_id (int): 当前帧的编号；
* prev_gray (numpy.ndarray): 前一帧输入网络图像的灰度图；
* prev_cfd (numpy.ndarray): 前一帧光流追踪图和预测结果融合图
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
**返回**
* img_matting (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-1 (0为全透明，1为不透明)。
* cur_gray (numpy.ndarray): 当前帧输入网络图像的灰度图；
* optflow_map (numpy.ndarray): 当前帧光流追踪图和预测结果融合图
```python
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_mobile_video_result'):
```
预测API，用于视频人像分割。
**参数**
* video\_path (str): 待分割视频路径。若为None，则从本地摄像头获取视频，并弹出窗口显示在线分割结果。
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* save\_dir (str): 视频保存路径，仅在video\_path不为None时启用，保存离线视频处理结果。
```python
def save_inference_model(dirname='humanseg_mobile_model',
model_filename=None,
params_filename=None,
combined=True)
```
将模型保存到指定路径。
**参数**
* dirname: 存在模型的目录名称
* model\_filename: 模型文件名称，默认为\_\_model\_\_
* params\_filename: 参数文件名称，默认为\_\_params\_\_(仅当`combined`为True时生效)
* combined: 是否将参数保存到统一的一个文件中
## 代码示例
图片分割及视频分割代码示例：
```python
import cv2
import paddlehub as hub
human_seg = hub.Module('humanseg_mobile')
im = cv2.imread('/PATH/TO/IMAGE')
#visualization=True可以用于查看超分图片效果，可设置为False提升运行速度。
res = human_seg.segment(images=[im],visualization=True)
print(res[0]['data'])
human_seg.video_segment('/PATH/TO/VIDEO')
human_seg.save_inference_model('/PATH/TO/SAVE/MODEL')
```
视频流预测代码示例：
```python
import cv2
import numpy as np
import paddlehub as hub
human_seg = hub.Module('humanseg_mobile')
cap_video = cv2.VideoCapture('\PATH\TO\VIDEO')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'humanseg_mobile_video.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps, (width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = human_seg.video_stream_segment(frame_org=frame_org, frame_id=cap_video.get(1), prev_gray=prev_gray, prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
```
## 服务部署
PaddleHub Serving可以部署一个人像分割的在线服务。
## 第一步：启动PaddleHub Serving
运行启动命令：
```shell
$ hub serving start -m humanseg_mobile
```
这样就完成了一个人像分割的服务化API的部署，默认端口号为8866。
**NOTE:** 如使用GPU预测，则需要在启动服务之前，设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置。
## 第二步：发送预测请求
配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/humanseg_mobile"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 保存图片
mask =cv2.cvtColor(base64_to_cv2(r.json()["results"][0]['data']), cv2.COLOR_BGR2GRAY)
rgba = np.concatenate((org_im, np.expand_dims(mask, axis=2)), axis=2)
cv2.imwrite("segment_human_mobile.png", rgba)
```
### 查看代码
<https://github.com/PaddlePaddle/PaddleSeg/tree/develop/contrib/HumanSeg>
### 依赖
paddlepaddle >= 1.8.0
paddlehub >= 1.7.1
modules/image/Image_gan/humanseg/humanseg_mobile/data_feed.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
from collections import OrderedDict
import cv2
import numpy as np
__all__ = ['reader', 'preprocess_v']
def preprocess_v(img, w, h):
img = cv2.resize(img, (w, h), cv2.INTER_LINEAR).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
return img
def reader(images=None, paths=None):
"""
Preprocess to yield image.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C]
paths (list[str]): paths to images.
Yield:
each (collections.OrderedDict): info of original image, preprocessed image.
"""
component = list()
if paths:
for im_path in paths:
each = OrderedDict()
assert os.path.isfile(
im_path), "The {} isn't a valid file path.".format(im_path)
#print(im_path)
im = cv2.imread(im_path).astype('float32')
each['org_im'] = im
each['org_im_path'] = im_path
each['org_im_shape'] = im.shape
component.append(each)
if images is not None:
assert type(images) is list, "images should be a list."
for im in images:
each = OrderedDict()
each['org_im'] = im
each['org_im_path'] = 'ndarray_time={}'.format(
round(time.time(), 6) * 1e6)
each['org_im_shape'] = im.shape
component.append(each)
for element in component:
img = element['org_im'].copy()
img = cv2.resize(img, (192, 192)).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
element['image'] = img
yield element
modules/image/Image_gan/humanseg/humanseg_mobile/module.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import ast
import os
import os.path as osp
import argparse
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
from paddle.fluid.core import PaddleTensor, AnalysisConfig, create_paddle_predictor
from paddlehub.module.module import moduleinfo, runnable, serving
from humanseg_mobile.processor import postprocess, base64_to_cv2, cv2_to_base64, check_dir
from humanseg_mobile.data_feed import reader, preprocess_v
from humanseg_mobile.optimal import postprocess_v, threshold_mask
@moduleinfo(
name="humanseg_mobile",
type="CV/semantic_segmentation",
author="paddlepaddle",
author_email="",
summary="HRNet_w18_samll_v1 is a semantic segmentation model.",
version="1.1.0")
class HRNetw18samllv1humanseg(hub.Module):
def _initialize(self):
self.default_pretrained_model_path = os.path.join(
self.directory, "humanseg_mobile_inference")
self._set_config()
def _set_config(self):
"""
predictor config setting
"""
self.model_file_path = os.path.join(self.default_pretrained_model_path,
'__model__')
self.params_file_path = os.path.join(self.default_pretrained_model_path,
'__params__')
cpu_config = AnalysisConfig(self.model_file_path, self.params_file_path)
cpu_config.disable_glog_info()
cpu_config.disable_gpu()
self.cpu_predictor = create_paddle_predictor(cpu_config)
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
use_gpu = True
except:
use_gpu = False
if use_gpu:
gpu_config = AnalysisConfig(self.model_file_path,
self.params_file_path)
gpu_config.disable_glog_info()
gpu_config.enable_use_gpu(
memory_pool_init_size_mb=1000, device_id=0)
self.gpu_predictor = create_paddle_predictor(gpu_config)
def segment(self,
images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_mobile_output'):
"""
API for human segmentation.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C], the color space is BGR.
paths (list[str]): The paths of images.
batch_size (int): batch size.
use_gpu (bool): Whether to use gpu.
visualization (bool): Whether to save image or not.
output_dir (str): The path to store output images.
Returns:
res (list[dict]): each element in the list is a dict, the keys and values are:
save_path (str, optional): the path to save images. (Exists only if visualization is True)
data (numpy.ndarray): data of post processed image.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly."
"If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
# compatibility with older versions
all_data = list()
for yield_data in reader(images, paths):
all_data.append(yield_data)
total_num = len(all_data)
loop_num = int(np.ceil(total_num / batch_size))
res = list()
for iter_id in range(loop_num):
batch_data = list()
handle_id = iter_id * batch_size
for image_id in range(batch_size):
try:
batch_data.append(all_data[handle_id + image_id])
except:
pass
# feed batch image
batch_image = np.array([data['image'] for data in batch_data])
batch_image = PaddleTensor(batch_image.copy())
output = self.gpu_predictor.run([
batch_image
]) if use_gpu else self.cpu_predictor.run([batch_image])
output = output[1].as_ndarray()
output = np.expand_dims(output[:, 1, :, :], axis=1)
# postprocess one by one
for i in range(len(batch_data)):
out = postprocess(
data_out=output[i],
org_im=batch_data[i]['org_im'],
org_im_shape=batch_data[i]['org_im_shape'],
org_im_path=batch_data[i]['org_im_path'],
output_dir=output_dir,
visualization=visualization)
res.append(out)
return res
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
"""
API for human video segmentation.
Args:
frame_org (numpy.ndarray): frame data, shape of each is [H, W, C], the color space is BGR.
frame_id (int): index of the frame to be decoded.
prev_gray (numpy.ndarray): gray scale image of last frame, shape of each is [H, W]
prev_cfd (numpy.ndarray): fusion image from optical flow image and segment result, shape of each is [H, W]
use_gpu (bool): Whether to use gpu.
Returns:
img_matting (numpy.ndarray): data of segmentation mask.
cur_gray (numpy.ndarray): gray scale image of current frame, shape of each is [H, W]
optflow_map (numpy.ndarray): optical flow image of current frame, shape of each is [H, W]
"""
resize_h = 192
resize_w = 192
is_init = True
width = int(frame_org.shape[0])
height = int(frame_org.shape[1])
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
if frame_id == 1:
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
else:
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (height, width), cv2.INTER_LINEAR)
return [img_matting, cur_gray, optflow_map]
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_mobile_video_result'):
"""
API for human video segmentation.
Args:
video_path (str): The path to take the video under preprocess. If video_path is None, it will capture
the vedio from your camera.
use_gpu (bool): Whether to use gpu.
save_dir (str): The path to store output video.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly. "
"If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
resize_h = 192
resize_w = 192
if not video_path:
cap_video = cv2.VideoCapture(0)
else:
cap_video = cv2.VideoCapture(video_path)
if not cap_video.isOpened():
raise IOError("Error opening video stream or file, "
"--video_path whether existing: {}"
" or camera whether working".format(video_path))
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
is_init = True
fps = cap_video.get(cv2.CAP_PROP_FPS)
if video_path is not None:
print('Please wait. It is computing......')
if not osp.exists(save_dir):
os.makedirs(save_dir)
save_path = osp.join(save_dir, 'result' + '.avi')
cap_out = cv2.VideoWriter(
save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
else:
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cv2.imshow('HumanSegmentation', comb)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap_video.release()
def save_inference_model(self,
dirname='humanseg_mobile_model',
model_filename=None,
params_filename=None,
combined=True):
if combined:
model_filename = "__model__" if not model_filename else model_filename
params_filename = "__params__" if not params_filename else params_filename
place = fluid.CPUPlace()
exe = fluid.Executor(place)
program, feeded_var_names, target_vars = fluid.io.load_inference_model(
dirname=self.default_pretrained_model_path,
model_filename=model_filename,
params_filename=params_filename,
executor=exe)
fluid.io.save_inference_model(
dirname=dirname,
main_program=program,
executor=exe,
feeded_var_names=feeded_var_names,
target_vars=target_vars,
model_filename=model_filename,
params_filename=params_filename)
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.segment(images=images_decode, **kwargs)
results = [{
'data': cv2_to_base64(result['data'])
} for result in results]
return results
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.segment(
paths=[args.input_path],
batch_size=args.batch_size,
use_gpu=args.use_gpu,
output_dir=args.output_dir,
visualization=args.visualization)
if args.save_dir is not None:
check_dir(args.save_dir)
self.save_inference_model(args.save_dir)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--use_gpu',
type=ast.literal_eval,
default=False,
help="whether use GPU or not")
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='humanseg_mobile_output',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='humanseg_mobile_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=ast.literal_eval,
default=False,
help="whether to save output as images.")
self.arg_config_group.add_argument(
'--batch_size',
type=ast.literal_eval,
default=1,
help="batch size.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
if __name__ == "__main__":
m = HRNetw18samllv1humanseg()
img = cv2.imread('photo.jpg')
#res = m.segment(images=[img], visualization=True)
#print(res[0]['data'])
#m.video_segment('')
cap_video = cv2.VideoCapture('video_test.mp4')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'result_frame.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path,
cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = m.video_stream_segment(
frame_org=frame_org,
frame_id=cap_video.get(1),
prev_gray=prev_gray,
prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(
np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
modules/image/Image_gan/humanseg/humanseg_mobile/optimal.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import numpy as np
def human_seg_tracking(pre_gray, cur_gray, prev_cfd, dl_weights, disflow):
"""计算光流跟踪匹配点和光流图
输入参数:
pre_gray: 上一帧灰度图
cur_gray: 当前帧灰度图
prev_cfd: 上一帧光流图
dl_weights: 融合权重图
disflow: 光流数据结构
返回值:
is_track: 光流点跟踪二值图，即是否具有光流点匹配
track_cfd: 光流跟踪图
"""
check_thres = 8
h, w = pre_gray.shape[:2]
track_cfd = np.zeros_like(prev_cfd)
is_track = np.zeros_like(pre_gray)
flow_fw = disflow.calc(pre_gray, cur_gray, None)
flow_bw = disflow.calc(cur_gray, pre_gray, None)
flow_fw = np.round(flow_fw).astype(np.int)
flow_bw = np.round(flow_bw).astype(np.int)
y_list = np.array(range(h))
x_list = np.array(range(w))
yv, xv = np.meshgrid(y_list, x_list)
yv, xv = yv.T, xv.T
cur_x = xv + flow_fw[:, :, 0]
cur_y = yv + flow_fw[:, :, 1]
# 超出边界不跟踪
not_track = (cur_x < 0) + (cur_x >= w) + (cur_y < 0) + (cur_y >= h)
flow_bw[~not_track] = flow_bw[cur_y[~not_track], cur_x[~not_track]]
not_track += (np.square(flow_fw[:, :, 0] + flow_bw[:, :, 0]) +
np.square(flow_fw[:, :, 1] + flow_bw[:, :, 1])) >= check_thres
track_cfd[cur_y[~not_track], cur_x[~not_track]] = prev_cfd[~not_track]
is_track[cur_y[~not_track], cur_x[~not_track]] = 1
not_flow = np.all(
np.abs(flow_fw) == 0, axis=-1) * np.all(
np.abs(flow_bw) == 0, axis=-1)
dl_weights[cur_y[not_flow], cur_x[not_flow]] = 0.05
return track_cfd, is_track, dl_weights
def human_seg_track_fuse(track_cfd, dl_cfd, dl_weights, is_track):
"""光流追踪图和人像分割结构融合
输入参数:
track_cfd: 光流追踪图
dl_cfd: 当前帧分割结果
dl_weights: 融合权重图
is_track: 光流点匹配二值图
返回
cur_cfd: 光流跟踪图和人像分割结果融合图
"""
fusion_cfd = dl_cfd.copy()
is_track = is_track.astype(np.bool)
fusion_cfd[is_track] = dl_weights[is_track] * dl_cfd[is_track] + (
1 - dl_weights[is_track]) * track_cfd[is_track]
# 确定区域
index_certain = ((dl_cfd > 0.9) + (dl_cfd < 0.1)) * is_track
index_less01 = (dl_weights < 0.1) * index_certain
fusion_cfd[index_less01] = 0.3 * dl_cfd[index_less01] + 0.7 * track_cfd[
index_less01]
index_larger09 = (dl_weights >= 0.1) * index_certain
fusion_cfd[index_larger09] = 0.4 * dl_cfd[index_larger09] + 0.6 * track_cfd[
index_larger09]
return fusion_cfd
def threshold_mask(img, thresh_bg, thresh_fg):
dst = (img / 255.0 - thresh_bg) / (thresh_fg - thresh_bg)
dst[np.where(dst > 1)] = 1
dst[np.where(dst < 0)] = 0
return dst.astype(np.float32)
def postprocess_v(cur_gray, scoremap, prev_gray, pre_cfd, disflow, is_init):
"""光流优化
Args:
cur_gray : 当前帧灰度图
pre_gray : 前一帧灰度图
pre_cfd ：前一帧融合结果
scoremap : 当前帧分割结果
difflow : 光流
is_init : 是否第一帧
Returns:
fusion_cfd : 光流追踪图和预测结果融合图
"""
h, w = scoremap.shape
cur_cfd = scoremap.copy()
if is_init:
if h <= 64 or w <= 64:
disflow.setFinestScale(1)
elif h <= 160 or w <= 160:
disflow.setFinestScale(2)
else:
disflow.setFinestScale(3)
fusion_cfd = cur_cfd
else:
weights = np.ones((h, w), np.float32) * 0.3
track_cfd, is_track, weights = human_seg_tracking(
prev_gray, cur_gray, pre_cfd, weights, disflow)
fusion_cfd = human_seg_track_fuse(track_cfd, cur_cfd, weights, is_track)
return fusion_cfd
modules/image/Image_gan/humanseg/humanseg_mobile/processor.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
import base64
import cv2
import numpy as np
__all__ = ['cv2_to_base64', 'base64_to_cv2', 'postprocess']
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
def postprocess(data_out,
org_im,
org_im_shape,
org_im_path,
output_dir,
visualization,
thresh=120):
"""
Postprocess output of network. one image at a time.
Args:
data_out (numpy.ndarray): output of network.
org_im (numpy.ndarray): original image.
org_im_shape (list): shape pf original image.
org_im_path (list): path of riginal image.
output_dir (str): output directory to store image.
visualization (bool): whether to save image or not.
thresh (float): threshold.
Returns:
result (dict): The data of processed image.
"""
result = dict()
for logit in data_out:
logit = (logit * 255).astype(np.uint8)
logit = cv2.resize(logit, (org_im_shape[1], org_im_shape[0]))
rgba = np.concatenate((org_im, np.expand_dims(logit, axis=2)), axis=2)
if visualization:
check_dir(output_dir)
save_im_path = get_save_image_name(org_im, org_im_path, output_dir)
cv2.imwrite(save_im_path, rgba)
result['save_path'] = save_im_path
result['data'] = logit
else:
result['data'] = logit
return result
def check_dir(dir_path):
if not os.path.exists(dir_path):
os.makedirs(dir_path)
elif os.path.isfile(dir_path):
os.remove(dir_path)
os.makedirs(dir_path)
def get_save_image_name(org_im, org_im_path, output_dir):
"""
Get save image name from source image path.
"""
# name prefix of orginal image
org_im_name = os.path.split(org_im_path)[-1]
im_prefix = os.path.splitext(org_im_name)[0]
ext = '.png'
# save image path
save_im_path = os.path.join(output_dir, im_prefix + ext)
if os.path.exists(save_im_path):
save_im_path = os.path.join(
output_dir, im_prefix + 'time={}'.format(int(time.time())) + ext)
return save_im_path
modules/image/Image_gan/humanseg/humanseg_server/README.md 已删除 100644 → 0
浏览文件 @ e6b082e1
## 模型概述
高精度模型，适用于服务端GPU且背景复杂的人像场景， 模型结构为Deeplabv3+/Xcetion65, 模型大小为158M，网络结构如图：
<p align="center">
<img src="https://paddlehub.bj.bcebos.com/paddlehub-img/deeplabv3plus.png" hspace='10'/> <br />
</p>
## 命令行预测
```
hub run humanseg_server --input_path "/PATH/TO/IMAGE"
```
## API
```python
def segment(self,
images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_server_output'):
```
预测API，用于人像分割。
**参数**
* images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* paths (list\[str\]): 图片的路径；
* batch\_size (int): batch 的大小；
* use\_gpu (bool): 是否使用 GPU；
* visualization (bool): 是否将识别结果保存为图片文件；
* output\_dir (str): 图片的保存路径。
**返回**
* res (list\[dict\]): 识别结果的列表，列表中每一个元素为 dict，关键字有 'save\_path', 'data'，对应的取值为：
* save\_path (str, optional): 可视化图片的保存路径（仅当visualization=True时存在）；
* data (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-255 (0为全透明，255为不透明)，也即取值越大的像素点越可能为人体，取值越小的像素点越可能为背景。
```python
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
```
预测API，用于逐帧对视频人像分割。
**参数**
* frame_org (numpy.ndarray): 单帧图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* frame_id (int): 当前帧的编号；
* prev_gray (numpy.ndarray): 前一帧输入网络图像的灰度图；
* prev_cfd (numpy.ndarray): 前一帧光流追踪图和预测结果融合图；
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置。
**返回**
* img_matting (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-1 (0为全透明，1为不透明)；
* cur_gray (numpy.ndarray): 当前帧输入分割网络图像的灰度图；
* optflow_map (numpy.ndarray): 当前帧光流追踪图和预测结果融合图。
```python
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_server_video'):
```
预测API，用于视频人像分割。
**参数**
* video\_path (str): 待分割视频路径。若为None，则从本地摄像头获取视频，并弹出窗口显示在线分割结果；
* use\_gpu (bool): 是否使用GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* save\_dir (str): 视频保存路径，仅在video\_path不为None时启用，保存离线视频处理结果。
```python
def save_inference_model(dirname,
model_filename=None,
params_filename=None,
combined=True):
```
将模型保存到指定路径。
**参数**
* dirname: 存在模型的目录名称
* model\_filename: 模型文件名称，默认为\_\_model\_\_
* params\_filename: 参数文件名称，默认为\_\_params\_\_(仅当`combined`为True时生效)
* combined: 是否将参数保存到统一的一个文件中
## 代码示例
图片分割及视频分割代码示例：
```python
import cv2
import paddlehub as hub
human_seg = hub.Module('humanseg_server')
im = cv2.imread('/PATH/TO/IMAGE')
#visualization=True可以用于查看超分图片效果，可设置为False提升运行速度。
res = human_seg.segment(images=[im],visualization=True)
print(res[0]['data'])
human_seg.video_segment('/PATH/TO/VIDEO')
human_seg.save_inference_model('/PATH/TO/SAVE/MODEL')
```
视频流预测代码示例：
```python
import cv2
import numpy as np
import paddlehub as hub
human_seg = hub.Module('humanseg_server')
cap_video = cv2.VideoCapture('\PATH\TO\VIDEO')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'humanseg_server_video.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps, (width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = human_seg.video_stream_segment(frame_org=frame_org, frame_id=cap_video.get(1), prev_gray=prev_gray, prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
```
## 服务部署
PaddleHub Serving可以部署一个人像分割的在线服务。
## 第一步：启动PaddleHub Serving
运行启动命令：
```shell
$ hub serving start -m humanseg_server
```
这样就完成了一个人像分割的服务化API的部署，默认端口号为8866。
**NOTE:** 如使用GPU预测，则需要在启动服务之前，设置CUDA\_VISIBLE\_DEVICES环境变量，否则不用设置。
## 第二步：发送预测请求
配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
org_im = cv2.imread('PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/humanseg_server"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 保存图片
mask =cv2.cvtColor(base64_to_cv2(r.json()["results"][0]['data']), cv2.COLOR_BGR2GRAY)
rgba = np.concatenate((org_im, np.expand_dims(mask, axis=2)), axis=2)
cv2.imwrite("segment_human_server.png", rgba)
```
### 查看代码
https://github.com/PaddlePaddle/PaddleSeg/tree/develop/contrib/HumanSeg
### 依赖
paddlepaddle >= 1.8.0
paddlehub >= 1.7.1
modules/image/Image_gan/humanseg/humanseg_server/data_feed.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# coding=utf-8
import os
import time
from collections import OrderedDict
import cv2
import numpy as np
from PIL import Image
__all__ = ['reader', 'preprocess_v']
def preprocess_v(img, w, h):
img = cv2.resize(img, (w, h), cv2.INTER_LINEAR).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
return img
def reader(images=None, paths=None):
"""
Preprocess to yield image.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C]
paths (list[str]): paths to images.
Yield:
each (collections.OrderedDict): info of original image, preprocessed image.
"""
component = list()
if paths:
for im_path in paths:
each = OrderedDict()
assert os.path.isfile(
im_path), "The {} isn't a valid file path.".format(im_path)
im = cv2.imread(im_path).astype('float32')
each['org_im'] = im
each['org_im_path'] = im_path
each['org_im_shape'] = im.shape
component.append(each)
if images is not None:
assert type(images) is list, "images should be a list."
for im in images:
each = OrderedDict()
each['org_im'] = im
each['org_im_path'] = 'ndarray_time={}'.format(
round(time.time(), 6) * 1e6)
each['org_im_shape'] = im.shape
component.append(each)
for element in component:
img = element['org_im'].copy()
img = cv2.resize(img, (513, 513)).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
element['image'] = img
yield element
modules/image/Image_gan/humanseg/humanseg_server/module.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import ast
import os
import os.path as osp
import argparse
import cv2
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
from paddle.fluid.core import PaddleTensor, AnalysisConfig, create_paddle_predictor
from paddlehub.module.module import moduleinfo, runnable, serving
from humanseg_server.processor import postprocess, base64_to_cv2, cv2_to_base64, check_dir
from humanseg_server.data_feed import reader, preprocess_v
from humanseg_server.optimal import postprocess_v, threshold_mask
@moduleinfo(
name="humanseg_server",
type="CV/semantic_segmentation",
author="baidu-vis",
author_email="",
summary="DeepLabv3+ is a semantic segmentation model.",
version="1.1.0")
class DeeplabV3pXception65HumanSeg(hub.Module):
def _initialize(self):
self.default_pretrained_model_path = os.path.join(
self.directory, "humanseg_server_inference")
self._set_config()
def _set_config(self):
"""
predictor config setting
"""
self.model_file_path = os.path.join(self.default_pretrained_model_path,
'__model__')
self.params_file_path = os.path.join(self.default_pretrained_model_path,
'__params__')
cpu_config = AnalysisConfig(self.model_file_path, self.params_file_path)
cpu_config.disable_glog_info()
cpu_config.disable_gpu()
self.cpu_predictor = create_paddle_predictor(cpu_config)
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
use_gpu = True
except:
use_gpu = False
if use_gpu:
gpu_config = AnalysisConfig(self.model_file_path,
self.params_file_path)
gpu_config.disable_glog_info()
gpu_config.enable_use_gpu(
memory_pool_init_size_mb=1000, device_id=0)
self.gpu_predictor = create_paddle_predictor(gpu_config)
def segment(self,
images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=False,
output_dir='humanseg_server_output'):
"""
API for human segmentation.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C], the color space is BGR.
paths (list[str]): The paths of images.
batch_size (int): batch size.
use_gpu (bool): Whether to use gpu.
visualization (bool): Whether to save image or not.
output_dir (str): The path to store output images.
Returns:
res (list[dict]): each element in the list is a dict, the keys and values are:
save_path (str, optional): the path to save images. (Exists only if visualization is True)
data (numpy.ndarray): data of post processed image.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly. If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
# compatibility with older versions
all_data = list()
for yield_data in reader(images, paths):
all_data.append(yield_data)
total_num = len(all_data)
loop_num = int(np.ceil(total_num / batch_size))
res = list()
for iter_id in range(loop_num):
batch_data = list()
handle_id = iter_id * batch_size
for image_id in range(batch_size):
try:
batch_data.append(all_data[handle_id + image_id])
except:
pass
# feed batch image
batch_image = np.array([data['image'] for data in batch_data])
batch_image = PaddleTensor(batch_image.copy())
output = self.gpu_predictor.run([
batch_image
]) if use_gpu else self.cpu_predictor.run([batch_image])
output = output[1].as_ndarray()
output = np.expand_dims(output[:, 1, :, :], axis=1)
# postprocess one by one
for i in range(len(batch_data)):
out = postprocess(
data_out=output[i],
org_im=batch_data[i]['org_im'],
org_im_shape=batch_data[i]['org_im_shape'],
org_im_path=batch_data[i]['org_im_path'],
output_dir=output_dir,
visualization=visualization)
res.append(out)
return res
def video_stream_segment(self,
frame_org,
frame_id,
prev_gray,
prev_cfd,
use_gpu=False):
"""
API for human video segmentation.
Args:
frame_org (numpy.ndarray): frame data, shape of each is [H, W, C], the color space is BGR.
frame_id (int): index of the frame to be decoded.
prev_gray (numpy.ndarray): gray scale image of last frame, shape of each is [H, W]
prev_cfd (numpy.ndarray): fusion image from optical flow image and segment result, shape of each is [H, W]
use_gpu (bool): Whether to use gpu.
Returns:
img_matting (numpy.ndarray): data of segmentation mask.
cur_gray (numpy.ndarray): gray scale image of current frame, shape of each is [H, W]
optflow_map (numpy.ndarray): optical flow image of current frame, shape of each is [H, W]
"""
resize_h = 512
resize_w = 512
is_init = True
width = int(frame_org.shape[0])
height = int(frame_org.shape[1])
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
if frame_id == 1:
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
else:
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (height, width), cv2.INTER_LINEAR)
return [img_matting, cur_gray, optflow_map]
def video_segment(self,
video_path=None,
use_gpu=False,
save_dir='humanseg_server_video'):
resize_h = 512
resize_w = 512
if not video_path:
cap_video = cv2.VideoCapture(0)
else:
cap_video = cv2.VideoCapture(video_path)
if not cap_video.isOpened():
raise IOError("Error opening video stream or file, "
"--video_path whether existing: {}"
" or camera whether working".format(video_path))
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
disflow = cv2.DISOpticalFlow_create(
cv2.DISOPTICAL_FLOW_PRESET_ULTRAFAST)
prev_gray = np.zeros((resize_h, resize_w), np.uint8)
prev_cfd = np.zeros((resize_h, resize_w), np.float32)
is_init = True
fps = cap_video.get(cv2.CAP_PROP_FPS)
if video_path is not None:
print('Please wait. It is computing......')
if not osp.exists(save_dir):
os.makedirs(save_dir)
save_path = osp.join(save_dir, 'result' + '.avi')
cap_out = cv2.VideoWriter(
save_path, cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
else:
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
frame = preprocess_v(frame_org, resize_w, resize_h)
image = PaddleTensor(np.array([frame.copy()]))
output = self.gpu_predictor.run(
[image]) if use_gpu else self.cpu_predictor.run([image])
score_map = output[1].as_ndarray()
frame = np.transpose(frame, axes=[1, 2, 0])
score_map = np.transpose(
np.squeeze(score_map, 0), axes=[1, 2, 0])
cur_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cur_gray = cv2.resize(cur_gray, (resize_w, resize_h))
score_map = 255 * score_map[:, :, 1]
optflow_map = postprocess_v(cur_gray, score_map, prev_gray,
prev_cfd, disflow, is_init)
prev_gray = cur_gray.copy()
prev_cfd = optflow_map.copy()
optflow_map = cv2.GaussianBlur(optflow_map, (3, 3), 0)
optflow_map = threshold_mask(
optflow_map, thresh_bg=0.2, thresh_fg=0.8)
img_matting = cv2.resize(optflow_map, (width, height),
cv2.INTER_LINEAR)
img_matting = np.repeat(
img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org +
(1 - img_matting) * bg_im).astype(np.uint8)
cv2.imshow('HumanSegmentation', comb)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap_video.release()
def save_inference_model(self,
dirname='humanseg_server_model',
model_filename=None,
params_filename=None,
combined=True):
if combined:
model_filename = "__model__" if not model_filename else model_filename
params_filename = "__params__" if not params_filename else params_filename
place = fluid.CPUPlace()
exe = fluid.Executor(place)
program, feeded_var_names, target_vars = fluid.io.load_inference_model(
dirname=self.default_pretrained_model_path,
model_filename=model_filename,
params_filename=params_filename,
executor=exe)
fluid.io.save_inference_model(
dirname=dirname,
main_program=program,
executor=exe,
feeded_var_names=feeded_var_names,
target_vars=target_vars,
model_filename=model_filename,
params_filename=params_filename)
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.segment(images=images_decode, **kwargs)
results = [{
'data': cv2_to_base64(result['data'])
} for result in results]
return results
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.segment(
paths=[args.input_path],
batch_size=args.batch_size,
use_gpu=args.use_gpu,
output_dir=args.output_dir,
visualization=args.visualization)
if args.save_dir is not None:
check_dir(args.save_dir)
self.save_inference_model(args.save_dir)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--use_gpu',
type=ast.literal_eval,
default=False,
help="whether use GPU or not")
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='humanseg_server_output',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='humanseg_server_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=ast.literal_eval,
default=False,
help="whether to save output as images.")
self.arg_config_group.add_argument(
'--batch_size',
type=ast.literal_eval,
default=1,
help="batch size.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
if __name__ == "__main__":
m = DeeplabV3pXception65HumanSeg()
# img = cv2.imread('photo.jpg')
# res = m.segment(images=[img])
# print(res[0]['data'])
# m.save_inference_model()
#m.video_segment(video_path='video_test.mp4')
img = cv2.imread('photo.jpg')
# res = m.segment(images=[img], visualization=True)
# print(res[0]['data'])
# m.video_segment('')
cap_video = cv2.VideoCapture('video_test.mp4')
fps = cap_video.get(cv2.CAP_PROP_FPS)
save_path = 'result_frame.avi'
width = int(cap_video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap_video.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap_out = cv2.VideoWriter(save_path,
cv2.VideoWriter_fourcc('M', 'J', 'P', 'G'), fps,
(width, height))
prev_gray = None
prev_cfd = None
while cap_video.isOpened():
ret, frame_org = cap_video.read()
if ret:
[img_matting, prev_gray, prev_cfd] = m.video_stream_segment(
frame_org=frame_org,
frame_id=cap_video.get(1),
prev_gray=prev_gray,
prev_cfd=prev_cfd)
img_matting = np.repeat(img_matting[:, :, np.newaxis], 3, axis=2)
bg_im = np.ones_like(img_matting) * 255
comb = (img_matting * frame_org + (1 - img_matting) * bg_im).astype(
np.uint8)
cap_out.write(comb)
else:
break
cap_video.release()
cap_out.release()
modules/image/Image_gan/humanseg/humanseg_server/optimal.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import numpy as np
def human_seg_tracking(pre_gray, cur_gray, prev_cfd, dl_weights, disflow):
"""计算光流跟踪匹配点和光流图
输入参数:
pre_gray: 上一帧灰度图
cur_gray: 当前帧灰度图
prev_cfd: 上一帧光流图
dl_weights: 融合权重图
disflow: 光流数据结构
返回值:
is_track: 光流点跟踪二值图，即是否具有光流点匹配
track_cfd: 光流跟踪图
"""
check_thres = 8
h, w = pre_gray.shape[:2]
track_cfd = np.zeros_like(prev_cfd)
is_track = np.zeros_like(pre_gray)
flow_fw = disflow.calc(pre_gray, cur_gray, None)
flow_bw = disflow.calc(cur_gray, pre_gray, None)
flow_fw = np.round(flow_fw).astype(np.int)
flow_bw = np.round(flow_bw).astype(np.int)
y_list = np.array(range(h))
x_list = np.array(range(w))
yv, xv = np.meshgrid(y_list, x_list)
yv, xv = yv.T, xv.T
cur_x = xv + flow_fw[:, :, 0]
cur_y = yv + flow_fw[:, :, 1]
# 超出边界不跟踪
not_track = (cur_x < 0) + (cur_x >= w) + (cur_y < 0) + (cur_y >= h)
flow_bw[~not_track] = flow_bw[cur_y[~not_track], cur_x[~not_track]]
not_track += (np.square(flow_fw[:, :, 0] + flow_bw[:, :, 0]) +
np.square(flow_fw[:, :, 1] + flow_bw[:, :, 1])) >= check_thres
track_cfd[cur_y[~not_track], cur_x[~not_track]] = prev_cfd[~not_track]
is_track[cur_y[~not_track], cur_x[~not_track]] = 1
not_flow = np.all(
np.abs(flow_fw) == 0, axis=-1) * np.all(
np.abs(flow_bw) == 0, axis=-1)
dl_weights[cur_y[not_flow], cur_x[not_flow]] = 0.05
return track_cfd, is_track, dl_weights
def human_seg_track_fuse(track_cfd, dl_cfd, dl_weights, is_track):
"""光流追踪图和人像分割结构融合
输入参数:
track_cfd: 光流追踪图
dl_cfd: 当前帧分割结果
dl_weights: 融合权重图
is_track: 光流点匹配二值图
返回
cur_cfd: 光流跟踪图和人像分割结果融合图
"""
fusion_cfd = dl_cfd.copy()
is_track = is_track.astype(np.bool)
fusion_cfd[is_track] = dl_weights[is_track] * dl_cfd[is_track] + (
1 - dl_weights[is_track]) * track_cfd[is_track]
# 确定区域
index_certain = ((dl_cfd > 0.9) + (dl_cfd < 0.1)) * is_track
index_less01 = (dl_weights < 0.1) * index_certain
fusion_cfd[index_less01] = 0.3 * dl_cfd[index_less01] + 0.7 * track_cfd[
index_less01]
index_larger09 = (dl_weights >= 0.1) * index_certain
fusion_cfd[index_larger09] = 0.4 * dl_cfd[index_larger09] + 0.6 * track_cfd[
index_larger09]
return fusion_cfd
def threshold_mask(img, thresh_bg, thresh_fg):
dst = (img / 255.0 - thresh_bg) / (thresh_fg - thresh_bg)
dst[np.where(dst > 1)] = 1
dst[np.where(dst < 0)] = 0
return dst.astype(np.float32)
def postprocess_v(cur_gray, scoremap, prev_gray, pre_cfd, disflow, is_init):
"""光流优化
Args:
cur_gray : 当前帧灰度图
pre_gray : 前一帧灰度图
pre_cfd ：前一帧融合结果
scoremap : 当前帧分割结果
difflow : 光流
is_init : 是否第一帧
Returns:
fusion_cfd : 光流追踪图和预测结果融合图
"""
h, w = scoremap.shape
cur_cfd = scoremap.copy()
if is_init:
if h <= 64 or w <= 64:
disflow.setFinestScale(1)
elif h <= 160 or w <= 160:
disflow.setFinestScale(2)
else:
disflow.setFinestScale(3)
fusion_cfd = cur_cfd
else:
weights = np.ones((h, w), np.float32) * 0.3
track_cfd, is_track, weights = human_seg_tracking(
prev_gray, cur_gray, pre_cfd, weights, disflow)
fusion_cfd = human_seg_track_fuse(track_cfd, cur_cfd, weights, is_track)
return fusion_cfd
modules/image/Image_gan/humanseg/humanseg_server/processor.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
import base64
import cv2
import numpy as np
__all__ = ['cv2_to_base64', 'base64_to_cv2', 'postprocess']
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
def postprocess(data_out, org_im, org_im_shape, org_im_path, output_dir,
visualization):
"""
Postprocess output of network. one image at a time.
Args:
data_out (numpy.ndarray): output of network.
org_im (numpy.ndarray): original image.
org_im_shape (list): shape pf original image.
org_im_path (list): path of riginal image.
output_dir (str): output directory to store image.
visualization (bool): whether to save image or not.
Returns:
result (dict): The data of processed image.
"""
result = dict()
for logit in data_out:
logit = (logit * 255).astype(np.uint8)
logit = cv2.resize(logit, (org_im_shape[1], org_im_shape[0]))
rgba = np.concatenate((org_im, np.expand_dims(logit, axis=2)), axis=2)
if visualization:
check_dir(output_dir)
save_im_path = get_save_image_name(org_im, org_im_path, output_dir)
cv2.imwrite(save_im_path, rgba)
result['save_path'] = save_im_path
result['data'] = rgba[:, :, 3]
else:
result['data'] = rgba[:, :, 3]
return result
def check_dir(dir_path):
if not os.path.exists(dir_path):
os.makedirs(dir_path)
elif os.path.isfile(dir_path):
os.remove(dir_path)
os.makedirs(dir_path)
def get_save_image_name(org_im, org_im_path, output_dir):
"""
Get save image name from source image path.
"""
# name prefix of orginal image
org_im_name = os.path.split(org_im_path)[-1]
im_prefix = os.path.splitext(org_im_name)[0]
ext = '.png'
# save image path
save_im_path = os.path.join(output_dir, im_prefix + ext)
if os.path.exists(save_im_path):
save_im_path = os.path.join(
output_dir, im_prefix + 'time={}'.format(int(time.time())) + ext)
return save_im_path
modules/image/Image_gan/semantic_segmentation/humanseg_lite/README.md 已删除 100644 → 0
浏览文件 @ e6b082e1
## 模型概述
HUmanSeg_lite是在ShuffleNetV2网络结构的基础上进行优化，进一步减小了网络规模，网络大小只有541K，量化后只有187K，适用于手机自拍人像分割，且能在移动端进行实时分割。
## 命令行预测
```
hub run humanseg_lite --input_path "/PATH/TO/IMAGE"
```
## API
```python
def segment(images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=True,
output_dir='humanseg_output')
```
预测API，用于人像分割。
**参数**
* images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* paths (list\[str\]): 图片的路径；
* batch\_size (int): batch 的大小；
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* visualization (bool): 是否将识别结果保存为图片文件；
* output\_dir (str): 图片的保存路径。
**返回**
* res (list\[dict\]): 识别结果的列表，列表中每一个元素为 dict，关键字有 'save\_path', 'data'，对应的取值为：
* save\_path (str, optional): 可视化图片的保存路径（仅当visualization=True时存在）；
* data (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-255 (0为全透明，255为不透明)，也即取值越大的像素点越可能为人体，取值越小的像素点越可能为背景。
```python
def save_inference_model(dirname,
model_filename=None,
params_filename=None,
combined=True)
```
将模型保存到指定路径。
**参数**
* dirname: 存在模型的目录名称
* model\_filename: 模型文件名称，默认为\_\_model\_\_
* params\_filename: 参数文件名称，默认为\_\_params\_\_(仅当`combined`为True时生效)
* combined: 是否将参数保存到统一的一个文件中
## 代码示例
```python
import cv2
import paddlehub as hub
human_seg = hub.Module('humanseg_lite')
im = cv2.imread('/PATH/TO/IMAGE')
res = human_seg.segment(images=[im],visualization=True)
```
## 服务部署
PaddleHub Serving可以部署一个人像分割的在线服务。
## 第一步：启动PaddleHub Serving
运行启动命令：
```shell
$ hub serving start -m humanseg_lite
```
这样就完成了一个人像分割的服务化API的部署，默认端口号为8866。
**NOTE:** 如使用GPU预测，则需要在启动服务之前，设置CUDA\_VISIBLE\_DEVICES环境变量，否则不用设置。
## 第二步：发送预测请求
配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
data = {'images':[cv2_to_base64(cv2.imread('/PATH/TO/IMAGE'))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/humanseg_lite"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 打印预测结果
print(base64_to_cv2(r.json()["results"][0]['data']))
```
### 依赖
paddlepaddle >= 1.8.1
paddlehub >= 1.7.1
modules/image/Image_gan/semantic_segmentation/humanseg_lite/data_feed.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
from collections import OrderedDict
import cv2
import numpy as np
from PIL import Image
__all__ = ['reader']
def reader(images=None, paths=None):
"""
Preprocess to yield image.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C]
paths (list[str]): paths to images.
Yield:
each (collections.OrderedDict): info of original image, preprocessed image.
"""
component = list()
if paths:
for im_path in paths:
each = OrderedDict()
assert os.path.isfile(
im_path), "The {} isn't a valid file path.".format(im_path)
#print(im_path)
im = cv2.imread(im_path).astype('float32')
each['org_im'] = im
each['org_im_path'] = im_path
each['org_im_shape'] = im.shape
component.append(each)
if images is not None:
assert type(images) is list, "images should be a list."
for im in images:
each = OrderedDict()
each['org_im'] = im
each['org_im_path'] = 'ndarray_time={}'.format(
round(time.time(), 6) * 1e6)
each['org_im_shape'] = im.shape
component.append(each)
for element in component:
img = element['org_im'].copy()
img = cv2.resize(img, (192, 192)).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
element['image'] = img
yield element
modules/image/Image_gan/semantic_segmentation/humanseg_lite/module.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import ast
import os
import argparse
import numpy as np
import paddle.fluid as fluid
import paddlehub as hub
from paddle.fluid.core import PaddleTensor, AnalysisConfig, create_paddle_predictor
from paddlehub.module.module import moduleinfo, runnable, serving
from humanseg_lite.processor import postprocess, base64_to_cv2, cv2_to_base64, check_dir
from humanseg_lite.data_feed import reader
@moduleinfo(
name="humanseg_lite",
type="CV/semantic_segmentation",
author="paddlepaddle",
author_email="",
summary="humanseg_lite is a semantic segmentation model.",
version="1.0.0")
class ShufflenetHumanSeg(hub.Module):
def _initialize(self):
self.default_pretrained_model_path = os.path.join(
self.directory, "humanseg_lite_inference")
self._set_config()
def _set_config(self):
"""
predictor config setting
"""
self.model_file_path = os.path.join(self.default_pretrained_model_path,
'__model__')
self.params_file_path = os.path.join(self.default_pretrained_model_path,
'__params__')
cpu_config = AnalysisConfig(self.model_file_path, self.params_file_path)
cpu_config.disable_glog_info()
cpu_config.disable_gpu()
self.cpu_predictor = create_paddle_predictor(cpu_config)
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
use_gpu = True
except:
use_gpu = False
if use_gpu:
gpu_config = AnalysisConfig(self.model_file_path,
self.params_file_path)
gpu_config.disable_glog_info()
gpu_config.enable_use_gpu(
memory_pool_init_size_mb=1000, device_id=0)
self.gpu_predictor = create_paddle_predictor(gpu_config)
def segment(self,
images=None,
paths=None,
data=None,
batch_size=1,
use_gpu=False,
visualization=True,
output_dir='humanseg_output'):
"""
API for human segmentation.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C], the color space is BGR.
paths (list[str]): The paths of images.
data (dict): key is 'image', the corresponding value is the path to image.
batch_size (int): batch size.
use_gpu (bool): Whether to use gpu.
visualization (bool): Whether to save image or not.
output_dir (str): The path to store output images.
Returns:
res (list[dict]): each element in the list is a dict, the keys and values are:
save_path (str, optional): the path to save images. (Exists only if visualization is True)
data (numpy.ndarray): data of post processed image.
"""
if use_gpu:
try:
_places = os.environ["CUDA_VISIBLE_DEVICES"]
int(_places[0])
except:
raise RuntimeError(
"Environment Variable CUDA_VISIBLE_DEVICES is not set correctly. If you wanna use gpu, please set CUDA_VISIBLE_DEVICES as cuda_device_id."
)
if data and 'image' in data:
if paths is None:
paths = list()
paths += data['image']
all_data = list()
for yield_data in reader(images, paths):
all_data.append(yield_data)
total_num = len(all_data)
loop_num = int(np.ceil(total_num / batch_size))
res = list()
for iter_id in range(loop_num):
batch_data = list()
handle_id = iter_id * batch_size
for image_id in range(batch_size):
try:
batch_data.append(all_data[handle_id + image_id])
except:
pass
# feed batch image
batch_image = np.array([data['image'] for data in batch_data])
batch_image = PaddleTensor(batch_image.copy())
output = self.gpu_predictor.run([
batch_image
]) if use_gpu else self.cpu_predictor.run([batch_image])
output = np.expand_dims(output[0].as_ndarray(), axis=1)
# postprocess one by one
for i in range(len(batch_data)):
out = postprocess(
data_out=output[i],
org_im=batch_data[i]['org_im'],
org_im_shape=batch_data[i]['org_im_shape'],
org_im_path=batch_data[i]['org_im_path'],
output_dir=output_dir,
visualization=visualization)
res.append(out)
return res
def save_inference_model(self,
dirname,
model_filename=None,
params_filename=None,
combined=True):
if combined:
model_filename = "__model__" if not model_filename else model_filename
params_filename = "__params__" if not params_filename else params_filename
place = fluid.CPUPlace()
exe = fluid.Executor(place)
program, feeded_var_names, target_vars = fluid.io.load_inference_model(
dirname=self.default_pretrained_model_path,
model_filename=model_filename,
params_filename=params_filename,
executor=exe)
fluid.io.save_inference_model(
dirname=dirname,
main_program=program,
executor=exe,
feeded_var_names=feeded_var_names,
target_vars=target_vars,
model_filename=model_filename,
params_filename=params_filename)
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.segment(images=images_decode, **kwargs)
results = [{
'data': cv2_to_base64(result['data'])
} for result in results]
return results
@runnable
def run_cmd(self, argvs):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(
title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options",
description=
"Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
args = self.parser.parse_args(argvs)
results = self.segment(
paths=[args.input_path],
batch_size=args.batch_size,
use_gpu=args.use_gpu,
output_dir=args.output_dir,
visualization=args.visualization)
if args.save_dir is not None:
check_dir(args.save_dir)
self.save_inference_model(args.save_dir)
return results
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument(
'--use_gpu',
type=ast.literal_eval,
default=False,
help="whether use GPU or not")
self.arg_config_group.add_argument(
'--output_dir',
type=str,
default='humanseg_output',
help="The directory to save output images.")
self.arg_config_group.add_argument(
'--save_dir',
type=str,
default='humanseg_model',
help="The directory to save model.")
self.arg_config_group.add_argument(
'--visualization',
type=ast.literal_eval,
default=True,
help="whether to save output as images.")
self.arg_config_group.add_argument(
'--batch_size',
type=ast.literal_eval,
default=1,
help="batch size.")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument(
'--input_path', type=str, help="path to image.")
if __name__ == "__main__":
m = ShufflenetHumanSeg()
import cv2
img = cv2.imread('./meditation.jpg')
res = m.segment(images=[img])
modules/image/Image_gan/semantic_segmentation/humanseg_lite/processor.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
import base64
import cv2
import numpy as np
__all__ = ['cv2_to_base64', 'base64_to_cv2', 'postprocess']
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
def postprocess(data_out,
org_im,
org_im_shape,
org_im_path,
output_dir,
visualization,
thresh=120):
"""
Postprocess output of network. one image at a time.
Args:
data_out (numpy.ndarray): output of network.
org_im (numpy.ndarray): original image.
org_im_shape (list): shape pf original image.
org_im_path (list): path of riginal image.
output_dir (str): output directory to store image.
visualization (bool): whether to save image or not.
thresh (float): threshold.
Returns:
result (dict): The data of processed image.
"""
result = dict()
for logit in data_out:
logit = np.squeeze(logit * 255, axis=2).astype(np.uint8)
logit = cv2.resize(logit, (org_im_shape[1], org_im_shape[0]))
rgba = np.concatenate((org_im, np.expand_dims(logit, axis=2)), axis=2)
if visualization:
check_dir(output_dir)
save_im_path = get_save_image_name(org_im, org_im_path, output_dir)
cv2.imwrite(save_im_path, rgba)
result['save_path'] = save_im_path
result['data'] = logit
else:
result['data'] = logit
return result
def check_dir(dir_path):
if not os.path.exists(dir_path):
os.makedirs(dir_path)
elif os.path.isfile(dir_path):
os.remove(dir_path)
os.makedirs(dir_path)
def get_save_image_name(org_im, org_im_path, output_dir):
"""
Get save image name from source image path.
"""
# name prefix of orginal image
org_im_name = os.path.split(org_im_path)[-1]
im_prefix = os.path.splitext(org_im_name)[0]
ext = '.png'
# save image path
save_im_path = os.path.join(output_dir, im_prefix + ext)
if os.path.exists(save_im_path):
save_im_path = os.path.join(
output_dir, im_prefix + 'time={}'.format(int(time.time())) + ext)
return save_im_path
modules/image/Image_gan/semantic_segmentation/humanseg_mobile/README.md 已删除 100644 → 0
浏览文件 @ e6b082e1
## 模型概述
HumanSeg-mobile是基于HRNet(Deep High-Resolution Representation Learning for Visual Recognition)的人像分割网络。HRNet在特征提取过程中保持了高分辨率的信息，保持了物体的细节信息，并可通过控制每个分支的通道数调整模型的大小。HumanSeg-mobile采用了HRNet_w18_small_v1的网络结构，模型大小只有5.8M， 适用于移动端或服务端CPU的前置摄像头场景。
## 命令行预测
```
hub run humanseg_mobile --input_path "/PATH/TO/IMAGE"
```
## API
```python
def segment(images=None,
paths=None,
batch_size=1,
use_gpu=False,
visualization=True,
output_dir='humanseg_output')
```
预测API，用于人像分割。
**参数**
* images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]，BGR格式；
* paths (list\[str\]): 图片的路径；
* batch\_size (int): batch 的大小；
* use\_gpu (bool): 是否使用 GPU预测，如果使用GPU预测，则在预测之前，请设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置；
* visualization (bool): 是否将识别结果保存为图片文件；
* output\_dir (str): 图片的保存路径。
**返回**
* res (list\[dict\]): 识别结果的列表，列表中每一个元素为 dict，关键字有 'save\_path', 'data'，对应的取值为：
* save\_path (str, optional): 可视化图片的保存路径（仅当visualization=True时存在）；
* data (numpy.ndarray): 人像分割结果，仅包含Alpha通道，取值为0-255 (0为全透明，255为不透明)，也即取值越大的像素点越可能为人体，取值越小的像素点越可能为背景。
```python
def save_inference_model(dirname,
model_filename=None,
params_filename=None,
combined=True)
```
将模型保存到指定路径。
**参数**
* dirname: 存在模型的目录名称
* model\_filename: 模型文件名称，默认为\_\_model\_\_
* params\_filename: 参数文件名称，默认为\_\_params\_\_(仅当`combined`为True时生效)
* combined: 是否将参数保存到统一的一个文件中
## 代码示例
```python
import cv2
import paddlehub as hub
human_seg = hub.Module('humanseg_mobile')
im = cv2.imread('/PATH/TO/IMAGE')
res = human_seg.segment(images=[im],visualization=True)
```
## 服务部署
PaddleHub Serving可以部署一个人像分割的在线服务。
## 第一步：启动PaddleHub Serving
运行启动命令：
```shell
$ hub serving start -m humanseg_mobile
```
这样就完成了一个人像分割的服务化API的部署，默认端口号为8866。
**NOTE:** 如使用GPU预测，则需要在启动服务之前，设置CUDA_VISIBLE_DEVICES环境变量，否则不用设置。
## 第二步：发送预测请求
配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# 发送HTTP请求
data = {'images':[cv2_to_base64(cv2.imread('/PATH/TO/IMAGE'))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/humanseg_mobile"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 打印预测结果
print(base64_to_cv2(r.json()["results"][0]['data']))
```
### 依赖
paddlepaddle >= 1.8.1
paddlehub >= 1.7.1
modules/image/Image_gan/semantic_segmentation/humanseg_mobile/data_feed.py 已删除 100644 → 0
浏览文件 @ e6b082e1
# -*- coding:utf-8 -*-
import os
import time
from collections import OrderedDict
import cv2
import numpy as np
__all__ = ['reader']
def reader(images=None, paths=None):
"""
Preprocess to yield image.
Args:
images (list(numpy.ndarray)): images data, shape of each is [H, W, C]
paths (list[str]): paths to images.
Yield:
each (collections.OrderedDict): info of original image, preprocessed image.
"""
component = list()
if paths:
for im_path in paths:
each = OrderedDict()
assert os.path.isfile(
im_path), "The {} isn't a valid file path.".format(im_path)
#print(im_path)
im = cv2.imread(im_path).astype('float32')
each['org_im'] = im
each['org_im_path'] = im_path
each['org_im_shape'] = im.shape
component.append(each)
if images is not None:
assert type(images) is list, "images should be a list."
for im in images:
each = OrderedDict()
each['org_im'] = im
each['org_im_path'] = 'ndarray_time={}'.format(
round(time.time(), 6) * 1e6)
each['org_im_shape'] = im.shape
component.append(each)
for element in component:
img = element['org_im'].copy()
img = cv2.resize(img, (192, 192)).astype(np.float32)
img_mean = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img_std = np.array([0.5, 0.5, 0.5]).reshape((3, 1, 1))
img = img.transpose((2, 0, 1)) / 255
img -= img_mean
img /= img_std
element['image'] = img
yield element
paddlehub/serving/app_single.py 已删除 100644 → 0
浏览文件 @ e6b082e1
此差异已折叠。
paddlehub/tokenizer/tokenizer.py 已删除 100644 → 0
浏览文件 @ e6b082e1
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或