Skip to content

P
PaddleHub

PaddlePaddle / PaddleHub
接近 2 年 前同步成功

通知 284
Star 12117
Fork 2091
P
PaddleHub
未验证 提交 9f622a30 编写于 作者: K KP 提交者： GitHub
浏览文件
操作

Merge branch 'develop' into add_styleganv2mixing_module

上级 200c6132 c66be094
展开全部 显示空白变更内容
内联 并排
Showing with 8977 addition and 250 deletion
.pre-commit-config.yaml
浏览文件 @ 9f622a30
......@@ -30,3 +30,9 @@
- --show-source
- --statistics
files: \.py$
- repo: https://github.com/asottile/reorder_python_imports
rev: v2.4.0
hooks:
- id: reorder-python-imports
exclude: (?=third_party).*(\.py)$
README.md
浏览文件 @ 9f622a30
......@@ -4,7 +4,7 @@ English | [简体中文](README_ch.md)
<img src="./docs/imgs/paddlehub_logo.jpg" align="middle">
<p align="center">
<div align="center">
<h3> <a href=#QuickStart> QuickStart </a> | <a href="https://paddlehub.readthedocs.io/en/release-v2.1"> Tutorial </a> | <a href="https://www.paddlepaddle.org.cn/hublist"> Models List </a> | <a href="https://www.paddlepaddle.org.cn/hub"> Demos </a> </h3>
<h3> <a href=#QuickStart> QuickStart </a> | <a href="https://paddlehub.readthedocs.io/en/release-v2.1"> Tutorial </a> | <a href="./modules"> Models List </a> | <a href="https://www.paddlepaddle.org.cn/hub"> Demos </a> </h3>
</div>
------------------------------------------------------------------------------------------
......@@ -28,7 +28,7 @@ English | [简体中文](README_ch.md)
## Introduction and Features
- **PaddleHub** aims to provide developers with rich, high-quality, and directly usable pre-trained models.
- **Abundant Pre-trained Models**: 300+ pre-trained models cover the 5 major categories, including Image, Text, Audio, Video, and Industrial application. All of them are free for download and offline usage.
- **Abundant Pre-trained Models**: 360+ pre-trained models cover the 5 major categories, including Image, Text, Audio, Video, and Industrial application. All of them are free for download and offline usage.
- **No Need for Deep Learning Background**: you can use AI models quickly and enjoy the dividends of the artificial intelligence era.
- **Quick Model Prediction**: model prediction can be realized through a few lines of scripts to quickly experience the model effect.
- **Model As Service**: one-line command to build deep learning model API service deployment capabilities.
......@@ -36,6 +36,7 @@ English | [简体中文](README_ch.md)
- **Cross-platform**: support Linux, Windows, MacOS and other operating systems.
### Recent updates
- **2021.12.22**，The v2.2.0 version is released. [1]More than 100 new models released，including dialog, speech, segmentation, OCR, text processing, GANs, and many other categories. The total number of pre-trained models reaches [**【360】**](https://www.paddlepaddle.org.cn/hublist). [2]Add an [indexed file](./modules/README.md) including useful information of pretrained models supported by PaddleHub. [3]Refactor README of pretrained models.
- **2021.05.12:** Add an open-domain dialogue system, i.e., [plato-mini](https://www.paddlepaddle.org.cn/hubdetail?name=plato-mini&en_category=TextGeneration), to make it easy to build a chatbot in wechat with the help of the wechaty, [See Demo](https://github.com/KPatr1ck/paddlehub-wechaty-demo)
- **2021.04.27:** The v2.1.0 version is released. [1] Add supports for five new models, including two high-precision semantic segmentation models based on VOC dataset and three voice classification models. [2] Enforce the transfer learning capabilities for image semantic segmentation, text semantic matching and voice classification on related datasets. [3] Add the export function APIs for two kinds of model formats, i.,e, ONNX and PaddleInference. [4] Add the support for [BentoML](https://github.com/bentoml/BentoML/), which is a cloud native framework for serving deployment. Users can easily serve pre-trained models from PaddleHub by following the [Tutorial notebooks](https://github.com/PaddlePaddle/PaddleHub/blob/release/v2.1/demo/serving/bentoml/cloud-native-model-serving-with-bentoml.ipynb). Also, see this announcement and [Release note](https://github.com/bentoml/BentoML/releases/tag/v0.12.1) from BentoML. (Many thanks to @[parano](https://github.com/parano) @[cqvu](https://github.com/cqvu) @[deehrlic](https://github.com/deehrlic) for contributing this feature in PaddleHub). [5] The total number of pre-trained models reaches **【300】**.
- **2021.02.18:** The v2.0.0 version is released, making model development and debugging easier, and the finetune task is more flexible and easy to use.The ability to transfer learning for visual tasks is fully upgraded, supporting various tasks such as image classification, image coloring, and style transfer; Transformer models such as BERT, ERNIE, and RoBERTa are upgraded to dynamic graphs, supporting Fine-Tune capabilities for text classification and sequence labeling; Optimize the Serving capability, support multi-card prediction, automatic load balancing, and greatly improve performance; the new automatic data enhancement capability Auto Augment can efficiently search for data enhancement strategy combinations suitable for data sets. 61 new word vector models were added, including 51 Chinese models and 10 English models; add 4 image segmentation models, 2 depth models, 7 image generation models, and 3 text generation models, the total number of pre-trained models reaches **【274】**.
......@@ -44,8 +45,8 @@ English | [简体中文](README_ch.md)
## Visualization Demo [[More]](./docs/docs_en/visualization.md)
### **Computer Vision (161 models)**
## Visualization Demo [[More]](./docs/docs_en/visualization.md) [[ModelList]](./modules)
### **[Computer Vision (212 models)](./modules#Image)**
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_all.gif" width = "530" height = "400" />
</div>
......@@ -53,7 +54,7 @@ English | [简体中文](README_ch.md)
- Many thanks to CopyRight@[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)、[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)、[PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN)、[AnimeGAN](https://github.com/TachibanaYoshino/AnimeGANv2)、[openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose)、[PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)、[Zhengxia Zou](https://github.com/jiupinjia/SkyAR)、[PaddleClas](https://github.com/PaddlePaddle/PaddleClas) for the pre-trained models, you can try to train your models with them.
### **Natural Language Processing (129 models)**
### **[Natural Language Processing (130 models)](./modules#Text)**
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_all.gif" width = "640" height = "240" />
</div>
......@@ -62,9 +63,37 @@ English | [简体中文](README_ch.md)
### Speech (3 models)
### [Speech (15 models)](./modules#Audio)
- ASR speech recognition algorithm, multiple algorithms are available.
- The speech recognition effect is as follows:
<div align="center">
<table>
<thead>
<tr>
<th width=250> Input Audio </th>
<th width=550> Recognition Result </th>
</tr>
</thead>
<tbody>
<tr>
<td align = "center">
<a href="https://paddlespeech.bj.bcebos.com/PaddleAudio/en.wav" rel="nofollow">
<img align="center" src="./docs/imgs/Readme_Related/audio_icon.png" width=250 ></a><br>
</td>
<td >I knocked at the door on the ancient side of the building.</td>
</tr>
<tr>
<td align = "center">
<a href="https://paddlespeech.bj.bcebos.com/PaddleAudio/zh.wav" rel="nofollow">
<img align="center" src="./docs/imgs/Readme_Related/audio_icon.png" width=250></a><br>
</td>
<td>我认为跑步最重要的就是给我带来了身体健康。</td>
</tr>
</tbody>
</table>
</div>
- TTS speech synthesis algorithm, multiple algorithms are available.
- Many thanks to CopyRight@[Parakeet](https://github.com/PaddlePaddle/Parakeet) for the pre-trained models, you can try to train your models with Parakeet.
- Input: `Life was like a box of chocolates, you never know what you're gonna get.`
- The synthesis effect is as follows:
<div align="center">
......@@ -95,7 +124,9 @@ English | [简体中文](README_ch.md)
</table>
</div>
### Video (8 models)
- Many thanks to CopyRight@[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech) for the pre-trained models, you can try to train your models with PaddleSpeech.
### [Video (8 models)](./modules#Video)
- Short video classification trained via large-scale video datasets, supports 3000+ tag types prediction for short Form Videos.
- Many thanks to CopyRight@[PaddleVideo](https://github.com/PaddlePaddle/PaddleVideo) for the pre-trained model, you can try to train your models with PaddleVideo.
- `Example: Input a short video of swimming, the algorithm can output the result of "swimming"`
......
README_ch.md
浏览文件 @ 9f622a30
......@@ -4,7 +4,7 @@
<img src="./docs/imgs/paddlehub_logo.jpg" align="middle">
<p align="center">
<div align="center">
<h3> <a href=#QuickStart> 快速开始 </a> | <a href="https://paddlehub.readthedocs.io/zh_CN/release-v2.1//"> 教程文档 </a> | <a href="https://www.paddlepaddle.org.cn/hublist"> 模型搜索 </a> | <a href="https://www.paddlepaddle.org.cn/hub"> 演示Demo </a>
<h3> <a href=#QuickStart> 快速开始 </a> | <a href="https://paddlehub.readthedocs.io/zh_CN/release-v2.1//"> 教程文档 </a> | <a href="./modules/README_ch.md"> 模型库 </a> | <a href="https://www.paddlepaddle.org.cn/hub"> 演示Demo </a>
</h3>
</div>
......@@ -30,7 +30,7 @@
## 简介与特性
- PaddleHub旨在为开发者提供丰富的、高质量的、直接可用的预训练模型
- **【模型种类丰富】**: 涵盖CV、NLP、Audio、Video、工业应用主流五大品类的 300+ 预训练模型，全部开源下载，离线可运行
- **【模型种类丰富】**: 涵盖CV、NLP、Audio、Video、工业应用主流五大品类的 **360+** 预训练模型，全部开源下载，离线可运行
- **【超低使用门槛】**：无需深度学习背景、无需数据与训练过程，可快速使用AI模型
- **【一键模型快速预测】**：通过一行命令行或者极简的Python API实现模型调用，可快速体验模型效果
- **【一键模型转服务化】**：一行命令，搭建深度学习模型API服务化部署能力
......@@ -38,6 +38,7 @@
- **【跨平台兼容性】**：可运行于Linux、Windows、MacOS等多种操作系统
## 近期更新
- **2021.12.22**，发布v2.2.0版本。【1】新增100+高质量模型，涵盖对话、语音处理、语义分割、文字识别、文本处理、图像生成等多个领域，预训练模型总量达到[**【360+】**](https://www.paddlepaddle.org.cn/hublist)；【2】新增模型[检索列表](./modules/README_ch.md)，包含模型名称、网络、数据集和使用场景等信息，快速定位用户所需的模型；【3】模型文档排版优化，呈现数据集、指标、模型大小等更多实用信息。
- **2021.05.12**，新增轻量级中文对话模型[plato-mini](https://www.paddlepaddle.org.cn/hubdetail?name=plato-mini&en_category=TextGeneration)，可以配合使用wechaty实现微信闲聊机器人，[参考demo](https://github.com/KPatr1ck/paddlehub-wechaty-demo)
- **2021.04.27**，发布v2.1.0版本。【1】新增基于VOC数据集的高精度语义分割模型2个，语音分类模型3个。【2】新增图像语义分割、文本语义匹配、语音分类等相关任务的Fine-Tune能力以及相关任务数据集;完善部署能力：【3】新增ONNX和PaddleInference等模型格式的导出功能。【4】新增[BentoML](https://github.com/bentoml/BentoML) 云原生服务化部署能力，可以支持统一的多框架模型管理和模型部署的工作流，[详细教程](https://github.com/PaddlePaddle/PaddleHub/blob/release/v2.1/demo/serving/bentoml/cloud-native-model-serving-with-bentoml.ipynb). 更多内容可以参考BentoML 最新 v0.12.1 [Releasenote](https://github.com/bentoml/BentoML/releases/tag/v0.12.1).（感谢@[parano](https://github.com/parano) @[cqvu](https://github.com/cqvu) @[deehrlic](https://github.com/deehrlic)）的贡献与支持。【5】预训练模型总量达到[**【300】**](https://www.paddlepaddle.org.cn/hublist)个。
- **2021.02.18**，发布v2.0.0版本，【1】模型开发调试更简单，finetune接口更加灵活易用。视觉类任务迁移学习能力全面升级，支持[图像分类](./demo/image_classification/README.md)、[图像着色](./demo/colorization/README.md)、[风格迁移](./demo/style_transfer/README.md)等多种任务；BERT、ERNIE、RoBERTa等Transformer类模型升级至动态图，支持[文本分类](./demo/text_classification/README.md)、[序列标注](./demo/sequence_labeling/README.md)的Fine-Tune能力；【2】优化服务化部署Serving能力，支持多卡预测、自动负载均衡，性能大幅度提升；【3】新增自动数据增强能力[Auto Augment](./demo/autoaug/README.md)，能高效地搜索适合数据集的数据增强策略组合。【4】新增[词向量模型](./modules/text/embedding)61个，其中包含中文模型51个，英文模型10个；新增[图像分割](./modules/thirdparty/image/semantic_segmentation)模型4个、[深度模型](./modules/thirdparty/image/depth_estimation)2个、[图像生成](./modules/thirdparty/image/Image_gan/style_transfer)模型7个、[文本生成](./modules/thirdparty/text/text_generation)模型3个。【5】预训练模型总量达到[**【274】**](https://www.paddlepaddle.org.cn/hublist) 个。
......@@ -47,9 +48,9 @@
## **精品模型效果展示[【更多】](./docs/docs_ch/visualization.md)**
## **精品模型效果展示[【更多】](./docs/docs_ch/visualization.md)[【模型库】](./modules/README_ch.md)**
### **图像类（161个）**
### **[图像类（212个）](./modules/README_ch.md#图像)**
- 包括图像分类、人脸检测、口罩检测、车辆检测、人脸/人体/手部关键点检测、人像分割、80+语言文本识别、图像超分/上色/动漫化等
<div align="center">
<img src="./docs/imgs/Readme_Related/Image_all.gif" width = "530" height = "400" />
......@@ -58,7 +59,7 @@
- 感谢CopyRight@[PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR)、[PaddleDetection](https://github.com/PaddlePaddle/PaddleDetection)、[PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN)、[AnimeGAN](https://github.com/TachibanaYoshino/AnimeGANv2)、[openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose)、[PaddleSeg](https://github.com/PaddlePaddle/PaddleSeg)、[Zhengxia Zou](https://github.com/jiupinjia/SkyAR)、[PaddleClas](https://github.com/PaddlePaddle/PaddleClas) 提供相关预训练模型，训练能力开放，欢迎体验。
### **文本类（129个）**
### **[文本类（130个）](./modules/README_ch.md#文本)**
- 包括中文分词、词性标注与命名实体识别、句法分析、AI写诗/对联/情话/藏头诗、中文的评论情感分析、中文色情文本审核等
<div align="center">
<img src="./docs/imgs/Readme_Related/Text_all.gif" width = "640" height = "240" />
......@@ -67,9 +68,37 @@
- 感谢CopyRight@[ERNIE](https://github.com/PaddlePaddle/ERNIE)、[LAC](https://github.com/baidu/LAC)、[DDParser](https://github.com/baidu/DDParser)提供相关预训练模型，训练能力开放，欢迎体验。
### **语音类（3个）**
### **[语音类（15个）](./modules/README_ch.md#语音)**
- ASR语音识别算法，多种算法可选
- 语音识别效果如下:
<div align="center">
<table>
<thead>
<tr>
<th width=250> Input Audio </th>
<th width=550> Recognition Result </th>
</tr>
</thead>
<tbody>
<tr>
<td align = "center">
<a href="https://paddlespeech.bj.bcebos.com/PaddleAudio/en.wav" rel="nofollow">
<img align="center" src="./docs/imgs/Readme_Related/audio_icon.png" width=250 ></a><br>
</td>
<td >I knocked at the door on the ancient side of the building.</td>
</tr>
<tr>
<td align = "center">
<a href="https://paddlespeech.bj.bcebos.com/PaddleAudio/zh.wav" rel="nofollow">
<img align="center" src="./docs/imgs/Readme_Related/audio_icon.png" width=250></a><br>
</td>
<td>我认为跑步最重要的就是给我带来了身体健康。</td>
</tr>
</tbody>
</table>
</div>
- TTS语音合成算法，多种算法可选
- 感谢CopyRight@[Parakeet](https://github.com/PaddlePaddle/Parakeet)提供预训练模型，训练能力开放，欢迎体验。
- 输入：`Life was like a box of chocolates, you never know what you're gonna get.`
- 合成效果如下:
<div align="center">
......@@ -100,7 +129,9 @@
</table>
</div>
### **视频类（8个）**
- 感谢CopyRight@[PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech)提供预训练模型，训练能力开放，欢迎体验。
### **[视频类（8个）](./modules/README_ch.md#视频)**
- 包含短视频分类，支持3000+标签种类，可输出TOP-K标签，多种算法可选。
- 感谢CopyRight@[PaddleVideo](https://github.com/PaddlePaddle/PaddleVideo)提供预训练模型，训练能力开放，欢迎体验。
- `举例：输入一段游泳的短视频，算法可以输出"游泳"结果`
......
modules/README.md
浏览文件 @ 9f622a30
此差异已折叠。
modules/README_ch.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/audio/asr/u2_conformer_aishell/README.md
浏览文件 @ 9f622a30
......@@ -3,7 +3,7 @@
|模型名称|u2_conformer_aishell|
| :--- | :---: |
|类别|语音-语音识别|
|网络|DeepSpeech2|
|网络|Conformer|
|数据集|AISHELL-1|
|是否支持Fine-tuning|否|
|模型大小|284MB|
......
modules/audio/asr/u2_conformer_librispeech/README.md
浏览文件 @ 9f622a30
......@@ -3,7 +3,7 @@
|模型名称|u2_conformer_librispeech|
| :--- | :---: |
|类别|语音-语音识别|
|网络|DeepSpeech2|
|网络|Conformer|
|数据集|LibriSpeech|
|是否支持Fine-tuning|否|
|模型大小|191MB|
......
modules/image/Image_editing/colorization/deoldify/README.md
浏览文件 @ 9f622a30
......@@ -53,9 +53,9 @@
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
- ```python
import paddlehub as hub
model = hub.Module(name='deoldify')
......
modules/image/Image_editing/colorization/deoldify/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# deoldify
| Module Name |deoldify|
| :--- | :---: |
|Category|Image editing|
|Network |NoGAN|
|Dataset|ILSVRC 2012|
|Fine-tuning supported or not |No|
|Module Size |834MB|
|Data indicators|-|
|Latest update date |2021-04-13|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/130886749-668dfa38-42ed-4a09-8d4a-b18af0475375.jpg" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130886685-76221736-839a-46a2-8415-e5e0dd3b345e.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- Deoldify is a color rendering model for images and videos, which can restore color for black and white photos and videos.
- For more information, please refer to: [deoldify](https://github.com/jantic/DeOldify)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- NOTE: This Module relies on ffmpeg, Please install ffmpeg before using this Module.
```shell
$ conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge
```
- ### 2、Installation
- ```shell
$ hub install deoldify
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import paddlehub as hub
model = hub.Module(name='deoldify')
model.predict('/PATH/TO/IMAGE/OR/VIDEO')
```
- ### 2、API
- ```python
def predict(self, input):
```
- Prediction API.
- **Parameter**
- input (str): Image path.
- **Return**
- If input is image path, the output is：
- pred_img(np.ndarray): image data, ndarray.shape is in the format [H, W, C], BGR.
- out_path(str): save path of images.
- If input is video path, the output is ：
- frame_pattern_combined(str): save path of frames from output video.
- vid_out_path(str): save path of output video.
- ```python
def run_image(self, img):
```
- Prediction API for image.
- **Parameter**
- img (str｜np.ndarray): Image data, str or ndarray. ndarray.shape is in the format [H, W, C], BGR.
- **Return**
- pred_img(np.ndarray): Ndarray.shape is in the format [H, W, C], BGR.
- ```python
def run_video(self, video):
```
- Prediction API for video.
- **Parameter**
- video(str): Video path.
- **Return**
- frame_pattern_combined(str): Save path of frames from output video.
- vid_out_path(str): Save path of output video.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of coloring old photos or videos.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m deoldify
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result.
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# Send an HTTP request
org_im = cv2.imread('/PATH/TO/ORIGIN/IMAGE')
data = {'images':cv2_to_base64(org_im)}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/deoldify"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
img = base64_to_cv2(r.json()["results"])
cv2.imwrite('/PATH/TO/SAVE/IMAGE', img)
```
## V. Release Note
- 1.0.0
First release
- 1.0.1
Adapt to paddlehub2.0
modules/image/Image_editing/colorization/photo_restoration/README.md
浏览文件 @ 9f622a30
......@@ -51,9 +51,9 @@
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
- ```python
import cv2
import paddlehub as hub
......
modules/image/Image_editing/colorization/photo_restoration/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# photo_restoration
|Module Name|photo_restoration|
| :--- | :---: |
|Category|Image editing|
|Network|deoldify and realsr|
|Fine-tuning supported or not|No|
|Module Size |64MB+834MB|
|Data indicators|-|
|Latest update date|2021-08-19|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/130897828-d0c86b81-63d1-4e9a-8095-bc000b8c7ca8.jpg" width = "260" height = "400" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130897762-5c9fa711-62bc-4067-8d44-f8feff8c574c.png" width = "260" height = "400" hspace='10'/>
</p>
- ### Module Introduction
- Photo_restoration can restore old photos. It mainly consists of two parts: coloring and super-resolution. The coloring model is deoldify
, and super resolution model is realsr. Therefore, when using this model, please install deoldify and realsr in advance.
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- NOTE: This Module relies on ffmpeg, Please install ffmpeg before using this Module.
```shell
$ conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge
```
- ### 2、Installation
- ```shell
$ hub install photo_restoration
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
model = hub.Module(name='photo_restoration', visualization=True)
im = cv2.imread('/PATH/TO/IMAGE')
res = model.run_image(im)
```
- ### 2、API
- ```python
def run_image(self,
input,
model_select= ['Colorization', 'SuperResolution'],
save_path = 'photo_restoration'):
```
- Predicition API, produce repaired photos.
- **Parameter**
- input (numpy.ndarray｜str): Image data，numpy.ndarray or str. ndarray.shape is in the format [H, W, C], BGR.
- model_select (list\[str\]): Mode selection，\['Colorization'\] only colorize the input image， \['SuperResolution'\] only increase the image resolution；
default is \['Colorization', 'SuperResolution'\]。
- save_path (str): Save path, default is 'photo_restoration'.
- **Return**
- output (numpy.ndarray): Restoration result，ndarray.shape is in the format [H, W, C], BGR.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of photo restoration.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m photo_restoration
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# Send an HTTP request
org_im = cv2.imread('PATH/TO/IMAGE')
data = {'images':cv2_to_base64(org_im), 'model_select': ['Colorization', 'SuperResolution']}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/photo_restoration"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
img = base64_to_cv2(r.json()["results"])
cv2.imwrite('PATH/TO/SAVE/IMAGE', img)
```
## V. Release Note
- 1.0.0
First release
- 1.0.1
Adapt to paddlehub2.0
modules/image/Image_editing/colorization/user_guided_colorization/README.md
浏览文件 @ 9f622a30
......@@ -22,7 +22,7 @@
<img src="https://user-images.githubusercontent.com/35907364/136653401-6644bd46-d280-4c15-8d48-680b7eb152cb.png" width = "300" height = "450" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/136648959-40493c9c-08ec-46cd-a2a2-5e2038dcbfa7.png" width = "300" height = "450" hspace='10'/>
</p>
- user_guided_colorization 是基于''Real-Time User-Guided Image Colorization with Learned Deep Priors"的着色模型，该模型利用预先提供的着色块对图像进行着色。
- user_guided_colorization 是基于"Real-Time User-Guided Image Colorization with Learned Deep Priors"的着色模型，该模型利用预先提供的着色块对图像进行着色。
## 二、安装
......
modules/image/Image_editing/colorization/user_guided_colorization/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# user_guided_colorization
|Module Name|user_guided_colorization|
| :--- | :---: |
|Category |Image editing|
|Network| Local and Global Hints Network |
|Dataset|ILSVRC 2012|
|Fine-tuning supported or notFine-tuning|Yes|
|Module Size|131MB|
|Data indicators|-|
|Latest update date |2021-02-26|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/136653401-6644bd46-d280-4c15-8d48-680b7eb152cb.png" width = "300" height = "450" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/136648959-40493c9c-08ec-46cd-a2a2-5e2038dcbfa7.png" width = "300" height = "450" hspace='10'/>
</p>
- ### Module Introduction
- User_guided_colorization is a colorization model based on "Real-Time User-Guided Image Colorization with Learned Deep Priors"，this model uses pre-supplied coloring blocks to color the gray image.
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install user_guided_colorization
```
- In case of any problems during installation, please refer to: [Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
```shell
$ hub run user_guided_colorization --input_path "/PATH/TO/IMAGE"
```
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
```python
import paddle
import paddlehub as hub
if __name__ == '__main__':
model = hub.Module(name='user_guided_colorization')
model.set_config(prob=0.1)
result = model.predict(images=['/PATH/TO/IMAGE'])
```
- ### 3.Fine-tune and Encapsulation
- After completing the installation of PaddlePaddle and PaddleHub, you can start using the user_guided_colorization model to fine-tune datasets such as [Canvas](../../docs/reference/datasets.md#class-hubdatasetsCanvas) by executing `python train.py`.
- Steps:
- Step1: Define the data preprocessing method
- ```python
import paddlehub.vision.transforms as T
transform = T.Compose([T.Resize((256, 256), interpolation='NEAREST'),
T.RandomPaddingCrop(crop_size=176),
T.RGB2LAB()], to_rgb=True)
```
- `transforms`: The data enhancement module defines lots of data preprocessing methods. Users can replace the data preprocessing methods according to their needs.
- Step2: Download the dataset
- ```python
from paddlehub.datasets import Canvas
color_set = Canvas(transform=transform, mode='train')
```
* `transforms`: Data preprocessing methods.
* `mode`: Select the data mode, the options are `train`, `test`, `val`. Default is `train`.
* `hub.datasets.Canvas()`: The dataset will be automatically downloaded from the network and decompressed to the `$HOME/.paddlehub/dataset` directory under the user directory.
- Step3: Load the pre-trained model
- ```python
model = hub.Module(name='user_guided_colorization', load_checkpoint=None)
model.set_config(classification=True, prob=1)
```
* `name`: Model name.
* `load_checkpoint`: Whether to load the self-trained model, if it is None, load the provided parameters.
* `classification`: The model is trained by two mode. At the beginning, `classification` is set to True, which is used for shallow network training. In the later stage of training, set `classification` to False, which is used to train the output layer of the network.
* `prob`: The probability that a priori color block is not added to each input image, the default is 1, that is, no prior color block is added. For example, when `prob` is set to 0.9, the probability that there are two a priori color blocks on a picture is(1-0.9)*(1-0.9)*0.9=0.009.
- Step4: Optimization strategy
```python
optimizer = paddle.optimizer.Adam(learning_rate=0.0001, parameters=model.parameters())
trainer = Trainer(model, optimizer, checkpoint_dir='img_colorization_ckpt_cls_1')
trainer.train(color_set, epochs=201, batch_size=25, eval_dataset=color_set, log_interval=10, save_interval=10)
```
- Run configuration
- `Trainer` mainly control the training of Fine-tune, including the following controllable parameters:
* `model`: Optimized model.
* `optimizer`: Optimizer selection.
* `use_vdl`: Whether to use vdl to visualize the training process.
* `checkpoint_dir`: The storage address of the model parameters.
* `compare_metrics`: The measurement index of the optimal model.
- `trainer.train` mainly control the specific training process, including the following controllable parameters:
* `train_dataset`: Training dataset.
* `epochs`: Epochs of training process.
* `batch_size`: Batch size.
* `num_workers`: Number of workers.
* `eval_dataset`: Validation dataset.
* `log_interval`:The interval for printing logs.
* `save_interval`: The interval for saving model parameters.
- Model prediction
- When Fine-tune is completed, the model with the best performance on the verification set will be saved in the `${CHECKPOINT_DIR}/best_model` directory. We use this model to make predictions. The `predict.py` script is as follows:
- ```python
import paddle
import paddlehub as hub
if __name__ == '__main__':
model = hub.Module(name='user_guided_colorization', load_checkpoint='/PATH/TO/CHECKPOINT')
model.set_config(prob=0.1)
result = model.predict(images=['/PATH/TO/IMAGE'])
```
- **NOTE:** If you want to get the oil painting style, please download the parameter file [Canvas colorization](https://paddlehub.bj.bcebos.com/dygraph/models/canvas_rc.pdparams)
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of colorization.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m user_guided_colorization
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import cv2
import base64
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
# Send an HTTP request
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/user_guided_colorization"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
data = base64_to_cv2(r.json()["results"]['data'][0]['fake_reg'])
cv2.imwrite('color.png', data)
```
## V. Release Note
* 1.0.0
First release
modules/image/Image_editing/super_resolution/dcscn/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# dcscn
|Module Name|dcscn|
| :--- | :---: |
|Category |Image editing|
|Network|dcscn|
|Dataset|DIV2k|
|Fine-tuning supported or not|No|
|Module Size|260KB|
|Data indicators|PSNR37.63|
|Data indicators |2021-02-26|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/133558583-0b7049db-ed1f-4a16-8676-f2141fcb3dee.png" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130899031-a6f8c58a-5cb7-4105-b990-8cca5ae15368.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- DCSCN is a super resolution model based on 'Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Network'. The model uses residual structure and skip connections to extract local and global features. It uses a parallel 1*1 convolutional network to learn detailed features to improve model performance. This model provides super resolution result with scale factor x2.
- For more information, please refer to: [dcscn](https://github.com/jiny2001/dcscn-super-resolution)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install dcscn
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```
$ hub run dcscn --input_path "/PATH/TO/IMAGE"
```
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
sr_model = hub.Module(name='dcscn')
im = cv2.imread('/PATH/TO/IMAGE').astype('float32')
res = sr_model.reconstruct(images=[im], visualization=True)
print(res[0]['data'])
sr_model.save_inference_model()
```
- ### 3、API
- ```python
def reconstruct(self,
images=None,
paths=None,
use_gpu=False,
visualization=False,
output_dir="dcscn_output")
```
- Prediction API.
- **Parameter**
* images (list\[numpy.ndarray\]): Image data，ndarray.shape is in the format \[H, W, C\]，BGR.
* paths (list\[str\]): image path.
* use\_gpu (bool): Use GPU or not. **set the CUDA_VISIBLE_DEVICES environment variable first if you are using GPU**.
* visualization (bool): Whether to save the recognition results as picture files.
* output\_dir (str): Save path of images, "dcscn_output" by default.
- **Return**
* res (list\[dict\]): The list of model results, where each element is dict and each field is:
* save\_path (str, optional): Save path of the result, save_path is '' if no image is saved.
* data (numpy.ndarray): Result of super resolution.
- ```python
def save_inference_model(self,
dirname='dcscn_save_model',
model_filename=None,
params_filename=None,
combined=False)
```
- Save the model to the specified path.
- **Parameters**
* dirname: Save path.
* model\_filename: Model file name，defalt is \_\_model\_\_
* params\_filename: Parameter file name，defalt is \_\_params\_\_(Only takes effect when `combined` is True)
* combined: Whether to save the parameters to a unified file.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of super resolution.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m dcscn
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/dcscn"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
sr = np.expand_dims(cv2.cvtColor(base64_to_cv2(r.json()["results"][0]['data']), cv2.COLOR_BGR2GRAY), axis=2)
shape =sr.shape
org_im = cv2.cvtColor(org_im, cv2.COLOR_BGR2YUV)
uv = cv2.resize(org_im[...,1:], (shape[1], shape[0]), interpolation=cv2.INTER_CUBIC)
combine_im = cv2.cvtColor(np.concatenate((sr, uv), axis=2), cv2.COLOR_YUV2BGR)
cv2.imwrite('dcscn_X2.png', combine_im)
print("save image as dcscn_X2.png")
```
## V. Release Note
- 1.0.0
First release
modules/image/Image_editing/super_resolution/falsr_a/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# falsr_a
|Module Name|falsr_a|
| :--- | :---: |
|Category |Image editing|
|Network |falsr_a|
|Dataset|DIV2k|
|Fine-tuning supported or not|No|
|Module Size |8.9MB|
|Data indicators|PSNR37.82|
|Latest update date|2021-02-26|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/133558583-0b7049db-ed1f-4a16-8676-f2141fcb3dee.png" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130899031-a6f8c58a-5cb7-4105-b990-8cca5ae15368.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- Falsr_a is a lightweight super-resolution model based on "Accurate and Lightweight Super-Resolution with Neural Architecture Search". The model uses a multi-objective approach to deal with the over-segmentation problem, and uses an elastic search strategy based on a hybrid controller to improve the performance of the model. This model provides super resolution result with scale factor x2.
- For more information, please refer to: [falsr_a](https://github.com/xiaomi-automl/FALSR)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install falsr_a
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```
$ hub run falsr_a --input_path "/PATH/TO/IMAGE"
```
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
sr_model = hub.Module(name='falsr_a')
im = cv2.imread('/PATH/TO/IMAGE').astype('float32')
res = sr_model.reconstruct(images=[im], visualization=True)
print(res[0]['data'])
sr_model.save_inference_model()
```
- ### 3、API
- ```python
def reconstruct(self,
images=None,
paths=None,
use_gpu=False,
visualization=False,
output_dir="falsr_a_output")
```
- Prediction API.
- **Parameter**
* images (list\[numpy.ndarray\]): image data，ndarray.shape is in the format \[H, W, C\]，BGR.
* paths (list\[str\]): image path.
* use\_gpu (bool): use GPU or not. **set the CUDA_VISIBLE_DEVICES environment variable first if you are using GPU**.
* visualization (bool): Whether to save the recognition results as picture files.
* output\_dir (str): save path of images, "dcscn_output" by default.
- **Return**
* res (list\[dict\]): The list of model results, where each element is dict and each field is:
* save\_path (str, optional): Save path of the result, save_path is '' if no image is saved.
* data (numpy.ndarray): result of super resolution.
- ```python
def save_inference_model(self,
dirname='falsr_a_save_model',
model_filename=None,
params_filename=None,
combined=False)
```
- Save the model to the specified path.
- **Parameters**
* dirname: Save path.
* model\_filename: model file name，defalt is \_\_model\_\_
* params\_filename: parameter file name，defalt is \_\_params\_\_(Only takes effect when `combined` is True)
* combined: Whether to save the parameters to a unified file.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of super resolution.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m falsr_a
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/falsr_a"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
sr = base64_to_cv2(r.json()["results"][0]['data'])
cv2.imwrite('falsr_a_X2.png', sr)
print("save image as falsr_a_X2.png")
```
## V. Release Note
- 1.0.0
First release
modules/image/Image_editing/super_resolution/falsr_b/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# falsr_b
|Module Name|falsr_b|
| :--- | :---: |
|Category |Image editing|
|Network |falsr_b|
|Dataset|DIV2k|
|Fine-tuning supported or not|No|
|Module Size |4MB|
|Data indicators|PSNR37.61|
|Latest update date|2021-02-26|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/133558583-0b7049db-ed1f-4a16-8676-f2141fcb3dee.png" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130899031-a6f8c58a-5cb7-4105-b990-8cca5ae15368.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- Falsr_b is a lightweight super-resolution model based on "Accurate and Lightweight Super-Resolution with Neural Architecture Search". The model uses a multi-objective approach to deal with the over-segmentation problem, and uses an elastic search strategy based on a hybrid controller to improve the performance of the model. This model provides super resolution result with scale factor x2.
- For more information, please refer to:[falsr_b](https://github.com/xiaomi-automl/FALSR)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install falsr_b
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```
$ hub run falsr_b --input_path "/PATH/TO/IMAGE"
```
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
```python
import cv2
import paddlehub as hub
sr_model = hub.Module(name='falsr_b')
im = cv2.imread('/PATH/TO/IMAGE').astype('float32')
res = sr_model.reconstruct(images=[im], visualization=True)
print(res[0]['data'])
sr_model.save_inference_model()
```
- ### 3、API
- ```python
def reconstruct(self,
images=None,
paths=None,
use_gpu=False,
visualization=False,
output_dir="falsr_b_output")
```
- Prediction API.
- **Parameter**
* images (list\[numpy.ndarray\]): Image data，ndarray.shape is in the format \[H, W, C\]，BGR.
* paths (list\[str\]): Image path.
* use\_gpu (bool): Use GPU or not. **set the CUDA_VISIBLE_DEVICES environment variable first if you are using GPU**.
* visualization (bool): Whether to save the recognition results as picture files.
* output\_dir (str): Save path of images, "dcscn_output" by default.
- **Return**
* res (list\[dict\]): The list of model results, where each element is dict and each field is:
* save\_path (str, optional): Save path of the result, save_path is '' if no image is saved.
* data (numpy.ndarray): Result of super resolution.
- ```python
def save_inference_model(self,
dirname='falsr_b_save_model',
model_filename=None,
params_filename=None,
combined=False)
```
- Save the model to the specified path.
- **Parameters**
* dirname: Save path.
* model\_filename: Model file name，defalt is \_\_model\_\_
* params\_filename: Parameter file name，defalt is \_\_params\_\_(Only takes effect when `combined` is True)
* combined: Whether to save the parameters to a unified file.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of super resolution.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m falsr_b
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/falsr_b"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
sr = base64_to_cv2(r.json()["results"][0]['data'])
cv2.imwrite('falsr_b_X2.png', sr)
print("save image as falsr_b_X2.png")
```
## V. Release Note
- 1.0.0
First release
modules/image/Image_editing/super_resolution/falsr_c/README.md
浏览文件 @ 9f622a30
......@@ -51,7 +51,7 @@
- ```
$ hub run falsr_c --input_path "/PATH/TO/IMAGE"
```
- ### 代码示例
- ### 2、预测代码示例
```python
import cv2
......@@ -65,7 +65,7 @@
sr_model.save_inference_model()
```
- ### 2、API
- ### 3、API
- ```python
def reconstruct(self,
......
modules/image/Image_editing/super_resolution/falsr_c/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# falsr_c
|Module Name|falsr_c|
| :--- | :---: |
|Category |Image editing|
|Network |falsr_c|
|Dataset|DIV2k|
|Fine-tuning supported or not|No|
|Module Size |4.4MB|
|Data indicators|PSNR37.66|
|Latest update date|2021-02-26|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/133558583-0b7049db-ed1f-4a16-8676-f2141fcb3dee.png" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130899031-a6f8c58a-5cb7-4105-b990-8cca5ae15368.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- Falsr_c is a lightweight super-resolution model based on "Accurate and Lightweight Super-Resolution with Neural Architecture Search". The model uses a multi-objective approach to deal with the over-segmentation problem, and uses an elastic search strategy based on a hybrid controller to improve the performance of the model. This model provides super resolution result with scale factor x2.
- For more information, please refer to:[falsr_c](https://github.com/xiaomi-automl/FALSR)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install falsr_c
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```
$ hub run falsr_c --input_path "/PATH/TO/IMAGE"
```
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
```python
import cv2
import paddlehub as hub
sr_model = hub.Module(name='falsr_c')
im = cv2.imread('/PATH/TO/IMAGE').astype('float32')
res = sr_model.reconstruct(images=[im], visualization=True)
print(res[0]['data'])
sr_model.save_inference_model()
```
- ### 3、API
- ```python
def reconstruct(self,
images=None,
paths=None,
use_gpu=False,
visualization=False,
output_dir="falsr_c_output")
```
- Prediction API.
- **Parameter**
* images (list\[numpy.ndarray\]): Image data，ndarray.shape is in the format \[H, W, C\]，BGR.
* paths (list\[str\]): Image path.
* use\_gpu (bool): Use GPU or not. **set the CUDA_VISIBLE_DEVICES environment variable first if you are using GPU**.
* visualization (bool): Whether to save the recognition results as picture files.
* output\_dir (str): Save path of images, "dcscn_output" by default.
- **Return**
* res (list\[dict\]): The list of model results, where each element is dict and each field is:
* save\_path (str, optional): Save path of the result, save_path is '' if no image is saved.
* data (numpy.ndarray): Result of super resolution.
- ```python
def save_inference_model(self,
dirname='falsr_c_save_model',
model_filename=None,
params_filename=None,
combined=False)
```
- Save the model to the specified path.
- **Parameters**
* dirname: Save path.
* model\_filename: Model file name，defalt is \_\_model\_\_
* params\_filename: Parameter file name，defalt is \_\_params\_\_(Only takes effect when `combined` is True)
* combined: Whether to save the parameters to a unified file.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of super resolution.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m falsr_c
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':[cv2_to_base64(org_im)]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/falsr_c"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
sr = base64_to_cv2(r.json()["results"][0]['data'])
cv2.imwrite('falsr_c_X2.png', sr)
print("save image as falsr_c_X2.png")
```
## V. Release Note
- 1.0.0
First release
modules/image/Image_editing/super_resolution/realsr/README.md
浏览文件 @ 9f622a30
......@@ -57,7 +57,7 @@
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
import paddlehub as hub
......
modules/image/Image_editing/super_resolution/realsr/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# realsr
|Module Name |reasr|
| :--- | :---: |
|Category |Image editing|
|Network|LP-KPN|
|Dataset |RealSR dataset|
|Fine-tuning supported or not|No|
|Module Size |64MB|
|Latest update date|2021-02-26|
|Data indicators |PSNR29.05|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/133558583-0b7049db-ed1f-4a16-8676-f2141fcb3dee.png" width = "450" height = "300" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/130789888-a0d4f78e-acd6-44c1-9570-7390e90ae8dc.png" width = "450" height = "300" hspace='10'/>
</p>
- ### Module Introduction
- Realsr is a super resolution model for image and video based on "Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Mode". This model provides super resolution result with scale factor x4.
- For more information, please refer to: [realsr](https://github.com/csjcai/RealSR)
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- **NOTE**: This Module relies on ffmpeg, Please install ffmpeg before using this Module.
```shell
$ conda install x264=='1!152.20180717' ffmpeg=4.0.2 -c conda-forge
```
- ### 2、Installation
- ```shell
$ hub install realsr
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import paddlehub as hub
model = hub.Module(name='realsr')
model.predict('/PATH/TO/IMAGE/OR/VIDEO')
```
- ### 2、API
- ```python
def predict(self, input):
```
- Prediction API.
- **Parameter**
- input (str): image path.
- **Return**
- If input is image path, the output is：
- pred_img(np.ndarray): image data, ndarray.shape is in the format [H, W, C], BGR.
- out_path(str): save path of images.
- If input is video path, the output is ：
- frame_pattern_combined(str): save path of frames from output video.
- vid_out_path(str): save path of output video.
- ```python
def run_image(self, img):
```
- Prediction API for images.
- **Parameter**
- img (str｜np.ndarray): Image data, str or ndarray. ndarray.shape is in the format [H, W, C], BGR.
- **Return**
- pred_img(np.ndarray): Prediction result, ndarray.shape is in the format [H, W, C], BGR.
- ```python
def run_video(self, video):
```
- Prediction API for video.
- **Parameter**
- video(str): Video path.
- **Return**
- frame_pattern_combined(str): Save path of frames from output video.
- vid_out_path(str): Save path of output video.
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of image super resolution.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m realsr
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import base64
import cv2
import numpy as np
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
org_im = cv2.imread('/PATH/TO/IMAGE')
data = {'images':cv2_to_base64(org_im)}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/realsr"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
img = base64_to_cv2(r.json()["results"])
cv2.imwrite('/PATH/TO/SAVE/IMAGE', img)
```
## V. Release Note
- 1.0.0
First release
* 1.0.1
Support paddlehub2.0
modules/image/Image_gan/attgan_celeba/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# attgan_celeba
|Module Name|attgan_celeba|
| :--- | :---: |
|Category |image generation|
|Network |AttGAN|
|Dataset|Celeba|
|Fine-tuning supported or not |No|
|Module Size |167MB|
|Latest update date|2021-02-26|
|Data indicators |-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/137855667-43c5c40c-28f5-45d8-accc-028e185b988f.JPG" width=1200><br/>
The image attributes are: original image, Bald, Bangs, Black_Hair, Blond_Hair, Brown_Hair, Bushy_Eyebrows, Eyeglasses, Gender, Mouth_Slightly_Open, Mustache, No_Beard, Pale_Skin, Aged<br/>
</p>
- ### Module Introduction
- AttGAN is a Generative Adversarial Network, which uses classification loss and reconstruction loss to train the network. The PaddleHub Module is trained one Celeba dataset and currently supports attributes of "Bald", "Bangs", "Black_Hair", "Blond_Hair", "Brown_Hair", "Bushy_Eyebrows", "Eyeglasses", "Gender", "Mouth_Slightly_Open", "Mustache", "No_Beard", "Pale_Skin", "Aged".
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.5.2
- paddlehub >= 1.0.0 | [How to install PaddleHub](../../../../docs/docs_en/get_start/installation.rst)
- ### 2、Installation
- ```shell
$ hub install attgan_celeba==1.0.0
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md).
## III. Module API Prediction
- ### 1、Command line Prediction
- ```shell
$ hub run attgan_celeba --image "/PATH/TO/IMAGE" --style "target_attribute"
```
- **Parameters**
- image: Input image path.
- style: Specify the attributes to be converted. The options are "Bald", "Bangs", "Black_Hair", "Blond_Hair", "Brown_Hair", "Bushy_Eyebrows", "Eyeglasses", "Gender", "Mouth_Slightly_Open", "Mustache", "No_Beard", "Pale_Skin", "Aged". You can choose one of the options.
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import paddlehub as hub
attgan = hub.Module(name="attgan_celeba")
test_img_path = ["/PATH/TO/IMAGE"]
trans_attr = ["Bangs"]
# set input dict
input_dict = {"image": test_img_path, "style": trans_attr}
# execute predict and print the result
results = attgan.generate(data=input_dict)
print(results)
```
- ### 3、API
- ```python
def generate(data)
```
- Style transfer API.
- **Parameter**
- data(list[dict]): Each element in the list is dict and each field is:
- image (list\[str\])： Each element in the list is the path of the image to be converted.
- style (list\[str\])： Each element in the list is a string, fill in the face attributes to be converted.
- **Return**
- res (list\[str\]): Save path of the result.
## IV. Release Note
- 1.0.0
First release
modules/image/Image_gan/cyclegan_cityscapes/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# cyclegan_cityscapes
|Module Name|cyclegan_cityscapes|
| :--- | :---: |
|Category |Image generation|
|Network |CycleGAN|
|Dataset|Cityscapes|
|Fine-tuning supported or not |No|
|Module Size |33MB|
|Latest update date |2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/137839740-4be4cf40-816f-401e-a73f-6cda037041dd.png" width = "450" height = "300" hspace='10'/>
<br />
Input image
<br />
<img src="https://user-images.githubusercontent.com/35907364/137839777-89fc705b-f0d7-4a93-94e2-76c0d3c5a0b0.png" width = "450" height = "300" hspace='10'/>
<br />
Output image
<br />
</p>
- ### Module Introduction
- CycleGAN belongs to Generative Adversarial Networks(GANs). Unlike traditional GANs that can only generate pictures in one direction, CycleGAN can simultaneously complete the style transfer of two domains. The PaddleHub Module is trained by Cityscapes dataset, and supports the conversion from real images to semantic segmentation results, and also supports conversion from semantic segmentation results to real images.
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.4.0
- paddlehub >= 1.1.0
- ### 2、Installation
- ```shell
$ hub install cyclegan_cityscapes==1.0.0
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```shell
$ hub run cyclegan_cityscapes --input_path "/PATH/TO/IMAGE"
```
- **Parameters**
- input_path: image path
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import paddlehub as hub
cyclegan = hub.Module(name="cyclegan_cityscapes")
test_img_path = "/PATH/TO/IMAGE"
# set input dict
input_dict = {"image": [test_img_path]}
# execute predict and print the result
results = cyclegan.generate(data=input_dict)
print(results)
```
- ### 3、API
- ```python
def generate(data)
```
- Style transfer API.
- **Parameters**
- data(list[dict]): Each element in the list is dict and each field is:
- image (list\[str\])： Image path.
- **Return**
- res (list\[str\]): The list of style transfer results, where each element is dict and each field is:
- origin: Original input path.
- generated: Save path of images.
## IV. Release Note
* 1.0.0
First release
modules/image/Image_gan/gan/first_order_motion/README.md 0 → 100644
浏览文件 @ 9f622a30
# first_order_motion
|模型名称|first_order_motion|
| :--- | :---: |
|类别|图像 - 图像生成|
|网络|S3FD|
|数据集|-|
|是否支持Fine-tuning|否|
|模型大小|343MB|
|最新更新日期|2021-12-24|
|数据指标|-|
## 一、模型基本信息
- ### 应用效果展示
- 样例结果示例：
<p align="center">
<img src="https://user-images.githubusercontent.com/22424850/147347145-1a7e84b6-2853-4490-8eaf-caf9cfdca79b.png" width = "40%" hspace='10'/>
<br />
输入图像
<br />
<img src="https://user-images.githubusercontent.com/22424850/147347151-d6c5690b-00cd-433f-b82b-3f8bb90bc7bd.gif" width = "40%" hspace='10'/>
<br />
输入视频
<br />
<img src="https://user-images.githubusercontent.com/22424850/147348127-52eb3f26-9b2c-49d5-a4a2-20a31f159802.gif" width = "40%" hspace='10'/>
<br />
输出视频
<br />
</p>
- ### 模型介绍
- First Order Motion的任务是图像动画/Image Animation，即输入为一张源图片和一个驱动视频，源图片中的人物则会做出驱动视频中的动作。
## 二、安装
- ### 1、环境依赖
- paddlepaddle >= 2.1.0
- paddlehub >= 2.1.0 | [如何安装PaddleHub](../../../../docs/docs_ch/get_start/installation.rst)
- ### 2、安装
- ```shell
$ hub install first_order_motion
```
- 如您安装时遇到问题，可参考：[零基础windows安装](../../../../docs/docs_ch/get_start/windows_quickstart.md)
| [零基础Linux安装](../../../../docs/docs_ch/get_start/linux_quickstart.md) | [零基础MacOS安装](../../../../docs/docs_ch/get_start/mac_quickstart.md)
## 三、模型API预测
- ### 1、命令行预测
- ```shell
$ hub run first_order_motion --source_image "/PATH/TO/IMAGE" --driving_video "/PATH/TO/VIDEO" --use_gpu
```
- 通过命令行方式实现视频驱动生成模型的调用，更多请见 [PaddleHub命令行指令](../../../../docs/docs_ch/tutorial/cmd_usage.rst)
- ### 2、预测代码示例
- ```python
import paddlehub as hub
module = hub.Module(name="first_order_motion")
module.generate(source_image="/PATH/TO/IMAGE", driving_video="/PATH/TO/VIDEO", ratio=0.4, image_size=256, output_dir='./motion_driving_result/', filename='result.mp4', use_gpu=False)
```
- ### 3、API
- ```python
generate(self, source_image=None, driving_video=None, ratio=0.4, image_size=256, output_dir='./motion_driving_result/', filename='result.mp4', use_gpu=False)
```
- 视频驱动生成API。
- **参数**
- source_image (str): 原始图片，支持单人图片和多人图片，视频中人物的表情动作将迁移到该原始图片中的人物上。
- driving_video (str): 驱动视频，视频中人物的表情动作作为待迁移的对象。
- ratio (float): 贴回驱动生成的人脸区域占原图的比例, 用户需要根据生成的效果调整该参数，尤其对于多人脸距离比较近的情况下需要调整改参数, 默认为0.4，调整范围是[0.4, 0.5]。
- image_size (int): 图片人脸大小，默认为256，可设置为512。
- output\_dir (str): 结果保存的文件夹名； <br/>
- filename (str): 结果保存的文件名。
- use\_gpu (bool): 是否使用 GPU；<br/>
## 四、更新历史
* 1.0.0
初始发布
- ```shell
$ hub install first_order_motion==1.0.0
```
modules/image/Image_gan/gan/first_order_motion/model.py 0 → 100644
浏览文件 @ 9f622a30
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
#
#Licensed under the Apache License, Version 2.0 (the "License");
#you may not use this file except in compliance with the License.
#You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#Unless required by applicable law or agreed to in writing, software
#distributed under the License is distributed on an "AS IS" BASIS,
#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#See the License for the specific language governing permissions and
#limitations under the License.
import os
import sys
import math
import pickle
import yaml
import imageio
import numpy as np
from tqdm import tqdm
from scipy.spatial import ConvexHull
import cv2
import paddle
from ppgan.utils.download import get_path_from_url
from ppgan.utils.animate import normalize_kp
from ppgan.modules.keypoint_detector import KPDetector
from ppgan.models.generators.occlusion_aware import OcclusionAwareGenerator
from ppgan.faceutils import face_detection
class FirstOrderPredictor:
def __init__(self,
weight_path=None,
config=None,
image_size=256,
relative=True,
adapt_scale=False,
find_best_frame=False,
best_frame=None,
face_detector='sfd',
multi_person=False,
face_enhancement=True,
batch_size=1,
mobile_net=False):
if config is not None and isinstance(config, str):
with open(config) as f:
self.cfg = yaml.load(f, Loader=yaml.SafeLoader)
elif isinstance(config, dict):
self.cfg = config
elif config is None:
self.cfg = {
'model': {
'common_params': {
'num_kp': 10,
'num_channels': 3,
'estimate_jacobian': True
},
'generator': {
'kp_detector_cfg': {
'temperature': 0.1,
'block_expansion': 32,
'max_features': 1024,
'scale_factor': 0.25,
'num_blocks': 5
},
'generator_cfg': {
'block_expansion': 64,
'max_features': 512,
'num_down_blocks': 2,
'num_bottleneck_blocks': 6,
'estimate_occlusion_map': True,
'dense_motion_params': {
'block_expansion': 64,
'max_features': 1024,
'num_blocks': 5,
'scale_factor': 0.25
}
}
}
}
}
self.image_size = image_size
if weight_path is None:
if mobile_net:
vox_cpk_weight_url = 'https://paddlegan.bj.bcebos.com/applications/first_order_model/vox-mobile.pdparams'
else:
if self.image_size == 512:
vox_cpk_weight_url = 'https://paddlegan.bj.bcebos.com/applications/first_order_model/vox-cpk-512.pdparams'
else:
vox_cpk_weight_url = 'https://paddlegan.bj.bcebos.com/applications/first_order_model/vox-cpk.pdparams'
weight_path = get_path_from_url(vox_cpk_weight_url)
self.weight_path = weight_path
self.relative = relative
self.adapt_scale = adapt_scale
self.find_best_frame = find_best_frame
self.best_frame = best_frame
self.face_detector = face_detector
self.generator, self.kp_detector = self.load_checkpoints(self.cfg, self.weight_path)
self.multi_person = multi_person
self.face_enhancement = face_enhancement
self.batch_size = batch_size
if face_enhancement:
from ppgan.faceutils.face_enhancement import FaceEnhancement
self.faceenhancer = FaceEnhancement(batch_size=batch_size)
def read_img(self, path):
img = imageio.imread(path)
if img.ndim == 2:
img = np.expand_dims(img, axis=2)
# som images have 4 channels
if img.shape[2] > 3:
img = img[:, :, :3]
return img
def run(self, source_image, driving_video, ratio, image_size, output_dir, filename):
self.ratio = ratio
self.image_size = image_size
self.output = output_dir
self.filename = filename
if not os.path.exists(output_dir):
os.makedirs(output_dir)
def get_prediction(face_image):
if self.find_best_frame or self.best_frame is not None:
i = self.best_frame if self.best_frame is not None else self.find_best_frame_func(
source_image, driving_video)
print("Best frame: " + str(i))
driving_forward = driving_video[i:]
driving_backward = driving_video[:(i + 1)][::-1]
predictions_forward = self.make_animation(
face_image,
driving_forward,
self.generator,
self.kp_detector,
relative=self.relative,
adapt_movement_scale=self.adapt_scale)
predictions_backward = self.make_animation(
face_image,
driving_backward,
self.generator,
self.kp_detector,
relative=self.relative,
adapt_movement_scale=self.adapt_scale)
predictions = predictions_backward[::-1] + predictions_forward[1:]
else:
predictions = self.make_animation(
face_image,
driving_video,
self.generator,
self.kp_detector,
relative=self.relative,
adapt_movement_scale=self.adapt_scale)
return predictions
source_image = self.read_img(source_image)
reader = imageio.get_reader(driving_video)
fps = reader.get_meta_data()['fps']
driving_video = []
try:
for im in reader:
driving_video.append(im)
except RuntimeError:
print("Read driving video error!")
pass
reader.close()
driving_video = [cv2.resize(frame, (self.image_size, self.image_size)) / 255.0 for frame in driving_video]
results = []
bboxes = self.extract_bbox(source_image.copy())
print(str(len(bboxes)) + " persons have been detected")
# for multi person
for rec in bboxes:
face_image = source_image.copy()[rec[1]:rec[3], rec[0]:rec[2]]
face_image = cv2.resize(face_image, (self.image_size, self.image_size)) / 255.0
predictions = get_prediction(face_image)
results.append({'rec': rec, 'predict': [predictions[i] for i in range(predictions.shape[0])]})
if len(bboxes) == 1 or not self.multi_person:
break
out_frame = []
for i in range(len(driving_video)):
frame = source_image.copy()
for result in results:
x1, y1, x2, y2, _ = result['rec']
h = y2 - y1
w = x2 - x1
out = result['predict'][i]
out = cv2.resize(out.astype(np.uint8), (x2 - x1, y2 - y1))
if len(results) == 1:
frame[y1:y2, x1:x2] = out
break
else:
patch = np.zeros(frame.shape).astype('uint8')
patch[y1:y2, x1:x2] = out
mask = np.zeros(frame.shape[:2]).astype('uint8')
cx = int((x1 + x2) / 2)
cy = int((y1 + y2) / 2)
cv2.circle(mask, (cx, cy), math.ceil(h * self.ratio), (255, 255, 255), -1, 8, 0)
frame = cv2.copyTo(patch, mask, frame)
out_frame.append(frame)
imageio.mimsave(os.path.join(self.output, self.filename), [frame for frame in out_frame], fps=fps)
def load_checkpoints(self, config, checkpoint_path):
generator = OcclusionAwareGenerator(
**config['model']['generator']['generator_cfg'], **config['model']['common_params'], inference=True)
kp_detector = KPDetector(**config['model']['generator']['kp_detector_cfg'], **config['model']['common_params'])
checkpoint = paddle.load(self.weight_path)
generator.set_state_dict(checkpoint['generator'])
kp_detector.set_state_dict(checkpoint['kp_detector'])
generator.eval()
kp_detector.eval()
return generator, kp_detector
def make_animation(self,
source_image,
driving_video,
generator,
kp_detector,
relative=True,
adapt_movement_scale=True):
with paddle.no_grad():
predictions = []
source = paddle.to_tensor(source_image[np.newaxis].astype(np.float32)).transpose([0, 3, 1, 2])
driving = paddle.to_tensor(np.array(driving_video).astype(np.float32)).transpose([0, 3, 1, 2])
kp_source = kp_detector(source)
kp_driving_initial = kp_detector(driving[0:1])
kp_source_batch = {}
kp_source_batch["value"] = paddle.tile(kp_source["value"], repeat_times=[self.batch_size, 1, 1])
kp_source_batch["jacobian"] = paddle.tile(kp_source["jacobian"], repeat_times=[self.batch_size, 1, 1, 1])
source = paddle.tile(source, repeat_times=[self.batch_size, 1, 1, 1])
begin_idx = 0
for frame_idx in tqdm(range(int(np.ceil(float(driving.shape[0]) / self.batch_size)))):
frame_num = min(self.batch_size, driving.shape[0] - begin_idx)
driving_frame = driving[begin_idx:begin_idx + frame_num]
kp_driving = kp_detector(driving_frame)
kp_source_img = {}
kp_source_img["value"] = kp_source_batch["value"][0:frame_num]
kp_source_img["jacobian"] = kp_source_batch["jacobian"][0:frame_num]
kp_norm = normalize_kp(
kp_source=kp_source,
kp_driving=kp_driving,
kp_driving_initial=kp_driving_initial,
use_relative_movement=relative,
use_relative_jacobian=relative,
adapt_movement_scale=adapt_movement_scale)
out = generator(source[0:frame_num], kp_source=kp_source_img, kp_driving=kp_norm)
img = np.transpose(out['prediction'].numpy(), [0, 2, 3, 1]) * 255.0
if self.face_enhancement:
img = self.faceenhancer.enhance_from_batch(img)
predictions.append(img)
begin_idx += frame_num
return np.concatenate(predictions)
def find_best_frame_func(self, source, driving):
import face_alignment
def normalize_kp(kp):
kp = kp - kp.mean(axis=0, keepdims=True)
area = ConvexHull(kp[:, :2]).volume
area = np.sqrt(area)
kp[:, :2] = kp[:, :2] / area
return kp
fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D, flip_input=True)
kp_source = fa.get_landmarks(255 * source)[0]
kp_source = normalize_kp(kp_source)
norm = float('inf')
frame_num = 0
for i, image in tqdm(enumerate(driving)):
kp_driving = fa.get_landmarks(255 * image)[0]
kp_driving = normalize_kp(kp_driving)
new_norm = (np.abs(kp_source - kp_driving)**2).sum()
if new_norm < norm:
norm = new_norm
frame_num = i
return frame_num
def extract_bbox(self, image):
detector = face_detection.FaceAlignment(
face_detection.LandmarksType._2D, flip_input=False, face_detector=self.face_detector)
frame = [image]
predictions = detector.get_detections_for_image(np.array(frame))
person_num = len(predictions)
if person_num == 0:
return np.array([])
results = []
face_boxs = []
h, w, _ = image.shape
for rect in predictions:
bh = rect[3] - rect[1]
bw = rect[2] - rect[0]
cy = rect[1] + int(bh / 2)
cx = rect[0] + int(bw / 2)
margin = max(bh, bw)
y1 = max(0, cy - margin)
x1 = max(0, cx - int(0.8 * margin))
y2 = min(h, cy + margin)
x2 = min(w, cx + int(0.8 * margin))
area = (y2 - y1) * (x2 - x1)
results.append([x1, y1, x2, y2, area])
# if a person has more than one bbox, keep the largest one
# maybe greedy will be better?
sorted(results, key=lambda area: area[4], reverse=True)
results_box = [results[0]]
for i in range(1, person_num):
num = len(results_box)
add_person = True
for j in range(num):
pre_person = results_box[j]
iou = self.IOU(pre_person[0], pre_person[1], pre_person[2], pre_person[3], pre_person[4], results[i][0],
results[i][1], results[i][2], results[i][3], results[i][4])
if iou > 0.5:
add_person = False
break
if add_person:
results_box.append(results[i])
boxes = np.array(results_box)
return boxes
def IOU(self, ax1, ay1, ax2, ay2, sa, bx1, by1, bx2, by2, sb):
#sa = abs((ax2 - ax1) * (ay2 - ay1))
#sb = abs((bx2 - bx1) * (by2 - by1))
x1, y1 = max(ax1, bx1), max(ay1, by1)
x2, y2 = min(ax2, bx2), min(ay2, by2)
w = x2 - x1
h = y2 - y1
if w < 0 or h < 0:
return 0.0
else:
return 1.0 * w * h / (sa + sb - w * h)
modules/image/Image_gan/gan/first_order_motion/module.py 0 → 100644
浏览文件 @ 9f622a30
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import argparse
import copy
import paddle
import paddlehub as hub
from paddlehub.module.module import moduleinfo, runnable, serving
import numpy as np
import cv2
from skimage.io import imread
from skimage.transform import rescale, resize
from .model import FirstOrderPredictor
@moduleinfo(
name="first_order_motion", type="CV/gan", author="paddlepaddle", author_email="", summary="", version="1.0.0")
class FirstOrderMotion:
def __init__(self):
self.pretrained_model = os.path.join(self.directory, "vox-cpk.pdparams")
self.network = FirstOrderPredictor(weight_path=self.pretrained_model, face_enhancement=True)
def generate(self,
source_image=None,
driving_video=None,
ratio=0.4,
image_size=256,
output_dir='./motion_driving_result/',
filename='result.mp4',
use_gpu=False):
'''
source_image (str): path to image<br/>
driving_video (str) : path to driving_video<br/>
ratio: margin ratio
image_size: size of image
output_dir: the dir to save the results
filename: filename to save the results
use_gpu: if True, use gpu to perform the computation, otherwise cpu.
'''
paddle.disable_static()
place = 'gpu:0' if use_gpu else 'cpu'
place = paddle.set_device(place)
if source_image == None or driving_video == None:
print('No image or driving video provided. Please input an image and a driving video.')
return
self.network.run(source_image, driving_video, ratio, image_size, output_dir, filename)
@runnable
def run_cmd(self, argvs: list):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options", description="Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
self.args = self.parser.parse_args(argvs)
self.generate(
source_image=self.args.source_image,
driving_video=self.args.driving_video,
ratio=self.args.ratio,
image_size=self.args.image_size,
output_dir=self.args.output_dir,
use_gpu=self.args.use_gpu)
return
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument('--use_gpu', action='store_true', help="use GPU or not")
self.arg_config_group.add_argument(
'--output_dir', type=str, default='motion_driving_result', help='output directory for saving result.')
self.arg_config_group.add_argument("--filename", default='result.mp4', help="filename to output")
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument("--source_image", type=str, help="path to source image")
self.arg_input_group.add_argument("--driving_video", type=str, help="path to driving video")
self.arg_input_group.add_argument("--ratio", dest="ratio", type=float, default=0.4, help="margin ratio")
self.arg_input_group.add_argument(
"--image_size", dest="image_size", type=int, default=256, help="size of image")
modules/image/Image_gan/gan/pixel2style2pixel/README.md 0 → 100644
浏览文件 @ 9f622a30
# pixel2style2pixel
|模型名称|pixel2style2pixel|
| :--- | :---: |
|类别|图像 - 图像生成|
|网络|Pixel2Style2Pixel|
|数据集|-|
|是否支持Fine-tuning|否|
|模型大小|1.7GB|
|最新更新日期|2021-12-14|
|数据指标|-|
## 一、模型基本信息
- ### 应用效果展示
- 样例结果示例：
<p align="center">
<img src="https://user-images.githubusercontent.com/22424850/146486444-63637926-4e46-4299-8905-d93f529d9d54.jpg" width = "40%" hspace='10'/>
<br />
输入图像
<br />
<img src="https://user-images.githubusercontent.com/22424850/146486413-0447dcc8-80ac-4b2c-8a7a-69347d60a2c4.png" width = "40%" hspace='10'/>
<br />
输出图像
<br />
</p>
- ### 模型介绍
- Pixel2Style2Pixel使用相当大的模型对图像进行编码，将图像编码到StyleGAN V2的风格向量空间中，使编码前的图像和解码后的图像具有强关联性。该模块应用于人脸转正任务。
## 二、安装
- ### 1、环境依赖
- paddlepaddle >= 2.1.0
- paddlehub >= 2.1.0 | [如何安装PaddleHub](../../../../docs/docs_ch/get_start/installation.rst)
- ### 2、安装
- ```shell
$ hub install pixel2style2pixel
```
- 如您安装时遇到问题，可参考：[零基础windows安装](../../../../docs/docs_ch/get_start/windows_quickstart.md)
| [零基础Linux安装](../../../../docs/docs_ch/get_start/linux_quickstart.md) | [零基础MacOS安装](../../../../docs/docs_ch/get_start/mac_quickstart.md)
## 三、模型API预测
- ### 1、命令行预测
- ```shell
# Read from a file
$ hub run pixel2style2pixel --input_path "/PATH/TO/IMAGE"
```
- 通过命令行方式实现人脸转正模型的调用，更多请见 [PaddleHub命令行指令](../../../../docs/docs_ch/tutorial/cmd_usage.rst)
- ### 2、预测代码示例
- ```python
import paddlehub as hub
module = hub.Module(name="pixel2style2pixel")
input_path = ["/PATH/TO/IMAGE"]
# Read from a file
module.style_transfer(paths=input_path, output_dir='./transfer_result/', use_gpu=True)
```
- ### 3、API
- ```python
style_transfer(images=None, paths=None, output_dir='./transfer_result/', use_gpu=False, visualization=True):
```
- 人脸转正生成API。
- **参数**
- images (list\[numpy.ndarray\]): 图片数据，ndarray.shape 为 \[H, W, C\]；<br/>
- paths (list\[str\]): 图片的路径；<br/>
- output\_dir (str): 结果保存的路径； <br/>
- use\_gpu (bool): 是否使用 GPU；<br/>
- visualization(bool): 是否保存结果到本地文件夹
## 四、服务部署
- PaddleHub Serving可以部署一个在线人脸转正服务。
- ### 第一步：启动PaddleHub Serving
- 运行启动命令：
- ```shell
$ hub serving start -m pixel2style2pixel
```
- 这样就完成了一个人脸转正的在线服务API的部署，默认端口号为8866。
- **NOTE:** 如使用GPU预测，则需要在启动服务之前，请设置CUDA\_VISIBLE\_DEVICES环境变量，否则不用设置。
- ### 第二步：发送预测请求
- 配置好服务端，以下数行代码即可实现发送预测请求，获取预测结果
- ```python
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
# 发送HTTP请求
data = {'images':[cv2_to_base64(cv2.imread("/PATH/TO/IMAGE"))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/pixel2style2pixel"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# 打印预测结果
print(r.json()["results"])
## 五、更新历史
* 1.0.0
初始发布
- ```shell
$ hub install pixel2style2pixel==1.0.0
```
modules/image/Image_gan/gan/pixel2style2pixel/model.py 0 → 100644
浏览文件 @ 9f622a30
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import cv2
import scipy
import random
import numpy as np
import paddle
import paddle.vision.transforms as T
import ppgan.faceutils as futils
from ppgan.models.generators import Pixel2Style2Pixel
from ppgan.utils.download import get_path_from_url
from PIL import Image
model_cfgs = {
'ffhq-inversion': {
'model_urls':
'https://paddlegan.bj.bcebos.com/models/pSp-ffhq-inversion.pdparams',
'transform':
T.Compose([T.Resize((256, 256)),
T.Transpose(),
T.Normalize([127.5, 127.5, 127.5], [127.5, 127.5, 127.5])]),
'size':
1024,
'style_dim':
512,
'n_mlp':
8,
'channel_multiplier':
2
},
'ffhq-toonify': {
'model_urls':
'https://paddlegan.bj.bcebos.com/models/pSp-ffhq-toonify.pdparams',
'transform':
T.Compose([T.Resize((256, 256)),
T.Transpose(),
T.Normalize([127.5, 127.5, 127.5], [127.5, 127.5, 127.5])]),
'size':
1024,
'style_dim':
512,
'n_mlp':
8,
'channel_multiplier':
2
},
'default': {
'transform':
T.Compose([T.Resize((256, 256)),
T.Transpose(),
T.Normalize([127.5, 127.5, 127.5], [127.5, 127.5, 127.5])])
}
}
def run_alignment(image):
img = Image.fromarray(image).convert("RGB")
face = futils.dlib.detect(img)
if not face:
raise Exception('Could not find a face in the given image.')
face_on_image = face[0]
lm = futils.dlib.landmarks(img, face_on_image)
lm = np.array(lm)[:, ::-1]
lm_eye_left = lm[36:42]
lm_eye_right = lm[42:48]
lm_mouth_outer = lm[48:60]
output_size = 1024
transform_size = 4096
enable_padding = True
# Calculate auxiliary vectors.
eye_left = np.mean(lm_eye_left, axis=0)
eye_right = np.mean(lm_eye_right, axis=0)
eye_avg = (eye_left + eye_right) * 0.5
eye_to_eye = eye_right - eye_left
mouth_left = lm_mouth_outer[0]
mouth_right = lm_mouth_outer[6]
mouth_avg = (mouth_left + mouth_right) * 0.5
eye_to_mouth = mouth_avg - eye_avg
# Choose oriented crop rectangle.
x = eye_to_eye - np.flipud(eye_to_mouth) * [-1, 1]
x /= np.hypot(*x)
x *= max(np.hypot(*eye_to_eye) * 2.0, np.hypot(*eye_to_mouth) * 1.8)
y = np.flipud(x) * [-1, 1]
c = eye_avg + eye_to_mouth * 0.1
quad = np.stack([c - x - y, c - x + y, c + x + y, c + x - y])
qsize = np.hypot(*x) * 2
# Shrink.
shrink = int(np.floor(qsize / output_size * 0.5))
if shrink > 1:
rsize = (int(np.rint(float(img.size[0]) / shrink)), int(np.rint(float(img.size[1]) / shrink)))
img = img.resize(rsize, Image.ANTIALIAS)
quad /= shrink
qsize /= shrink
# Crop.
border = max(int(np.rint(qsize * 0.1)), 3)
crop = (int(np.floor(min(quad[:, 0]))), int(np.floor(min(quad[:, 1]))), int(np.ceil(max(quad[:, 0]))),
int(np.ceil(max(quad[:, 1]))))
crop = (max(crop[0] - border, 0), max(crop[1] - border, 0), min(crop[2] + border, img.size[0]),
min(crop[3] + border, img.size[1]))
if crop[2] - crop[0] < img.size[0] or crop[3] - crop[1] < img.size[1]:
img = img.crop(crop)
quad -= crop[0:2]
# Pad.
pad = (int(np.floor(min(quad[:, 0]))), int(np.floor(min(quad[:, 1]))), int(np.ceil(max(quad[:, 0]))),
int(np.ceil(max(quad[:, 1]))))
pad = (max(-pad[0] + border, 0), max(-pad[1] + border, 0), max(pad[2] - img.size[0] + border, 0),
max(pad[3] - img.size[1] + border, 0))
if enable_padding and max(pad) > border - 4:
pad = np.maximum(pad, int(np.rint(qsize * 0.3)))
img = np.pad(np.float32(img), ((pad[1], pad[3]), (pad[0], pad[2]), (0, 0)), 'reflect')
h, w, _ = img.shape
y, x, _ = np.ogrid[:h, :w, :1]
mask = np.maximum(1.0 - np.minimum(np.float32(x) / pad[0],
np.float32(w - 1 - x) / pad[2]),
1.0 - np.minimum(np.float32(y) / pad[1],
np.float32(h - 1 - y) / pad[3]))
blur = qsize * 0.02
img += (scipy.ndimage.gaussian_filter(img, [blur, blur, 0]) - img) * np.clip(mask * 3.0 + 1.0, 0.0, 1.0)
img += (np.median(img, axis=(0, 1)) - img) * np.clip(mask, 0.0, 1.0)
img = Image.fromarray(np.uint8(np.clip(np.rint(img), 0, 255)), 'RGB')
quad += pad[:2]
# Transform.
img = img.transform((transform_size, transform_size), Image.QUAD, (quad + 0.5).flatten(), Image.BILINEAR)
return img
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super(AttrDict, self).__init__(*args, **kwargs)
self.__dict__ = self
class Pixel2Style2PixelPredictor:
def __init__(self,
weight_path=None,
model_type=None,
seed=None,
size=1024,
style_dim=512,
n_mlp=8,
channel_multiplier=2):
if weight_path is None and model_type != 'default':
if model_type in model_cfgs.keys():
weight_path = get_path_from_url(model_cfgs[model_type]['model_urls'])
size = model_cfgs[model_type].get('size', size)
style_dim = model_cfgs[model_type].get('style_dim', style_dim)
n_mlp = model_cfgs[model_type].get('n_mlp', n_mlp)
channel_multiplier = model_cfgs[model_type].get('channel_multiplier', channel_multiplier)
checkpoint = paddle.load(weight_path)
else:
raise ValueError('Predictor need a weight path or a pretrained model type')
else:
checkpoint = paddle.load(weight_path)
opts = checkpoint.pop('opts')
opts = AttrDict(opts)
opts['size'] = size
opts['style_dim'] = style_dim
opts['n_mlp'] = n_mlp
opts['channel_multiplier'] = channel_multiplier
self.generator = Pixel2Style2Pixel(opts)
self.generator.set_state_dict(checkpoint)
self.generator.eval()
if seed is not None:
paddle.seed(seed)
random.seed(seed)
np.random.seed(seed)
self.model_type = 'default' if model_type is None else model_type
def run(self, image):
src_img = run_alignment(image)
src_img = np.asarray(src_img)
transformed_image = model_cfgs[self.model_type]['transform'](src_img)
dst_img, latents = self.generator(
paddle.to_tensor(transformed_image[None, ...]), resize=False, return_latents=True)
dst_img = (dst_img * 0.5 + 0.5)[0].numpy() * 255
dst_img = dst_img.transpose((1, 2, 0))
dst_npy = latents[0].numpy()
return dst_img, dst_npy
modules/image/Image_gan/gan/pixel2style2pixel/module.py 0 → 100644
浏览文件 @ 9f622a30
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import argparse
import copy
import paddle
import paddlehub as hub
from paddlehub.module.module import moduleinfo, runnable, serving
import numpy as np
import cv2
from skimage.io import imread
from skimage.transform import rescale, resize
from .model import Pixel2Style2PixelPredictor
from .util import base64_to_cv2
@moduleinfo(
name="pixel2style2pixel",
type="CV/style_transfer",
author="paddlepaddle",
author_email="",
summary="",
version="1.0.0")
class pixel2style2pixel:
def __init__(self):
self.pretrained_model = os.path.join(self.directory, "pSp-ffhq-inversion.pdparams")
self.network = Pixel2Style2PixelPredictor(weight_path=self.pretrained_model, model_type='ffhq-inversion')
def style_transfer(self,
images=None,
paths=None,
output_dir='./transfer_result/',
use_gpu=False,
visualization=True):
'''
images (list[numpy.ndarray]): data of images, shape of each is [H, W, C], color space must be BGR(read by cv2).
paths (list[str]): paths to images
output_dir: the dir to save the results
use_gpu: if True, use gpu to perform the computation, otherwise cpu.
visualization: if True, save results in output_dir.
'''
results = []
paddle.disable_static()
place = 'gpu:0' if use_gpu else 'cpu'
place = paddle.set_device(place)
if images == None and paths == None:
print('No image provided. Please input an image or a image path.')
return
if images != None:
for image in images:
image = image[:, :, ::-1]
out = self.network.run(image)
results.append(out)
if paths != None:
for path in paths:
image = cv2.imread(path)[:, :, ::-1]
out = self.network.run(image)
results.append(out)
if visualization == True:
if not os.path.exists(output_dir):
os.makedirs(output_dir, exist_ok=True)
for i, out in enumerate(results):
if out is not None:
cv2.imwrite(os.path.join(output_dir, 'output_{}.png'.format(i)), out[0][:, :, ::-1])
np.save(os.path.join(output_dir, 'output_{}.npy'.format(i)), out[1])
return results
@runnable
def run_cmd(self, argvs: list):
"""
Run as a command.
"""
self.parser = argparse.ArgumentParser(
description="Run the {} module.".format(self.name),
prog='hub run {}'.format(self.name),
usage='%(prog)s',
add_help=True)
self.arg_input_group = self.parser.add_argument_group(title="Input options", description="Input data. Required")
self.arg_config_group = self.parser.add_argument_group(
title="Config options", description="Run configuration for controlling module behavior, not required.")
self.add_module_config_arg()
self.add_module_input_arg()
self.args = self.parser.parse_args(argvs)
results = self.style_transfer(
paths=[self.args.input_path],
output_dir=self.args.output_dir,
use_gpu=self.args.use_gpu,
visualization=self.args.visualization)
return results
@serving
def serving_method(self, images, **kwargs):
"""
Run as a service.
"""
images_decode = [base64_to_cv2(image) for image in images]
results = self.style_transfer(images=images_decode, **kwargs)
tolist = [result.tolist() for result in results]
return tolist
def add_module_config_arg(self):
"""
Add the command config options.
"""
self.arg_config_group.add_argument('--use_gpu', action='store_true', help="use GPU or not")
self.arg_config_group.add_argument(
'--output_dir', type=str, default='transfer_result', help='output directory for saving result.')
self.arg_config_group.add_argument('--visualization', type=bool, default=False, help='save results or not.')
def add_module_input_arg(self):
"""
Add the command input options.
"""
self.arg_input_group.add_argument('--input_path', type=str, help="path to input image.")
modules/image/Image_gan/gan/pixel2style2pixel/util.py 0 → 100644
浏览文件 @ 9f622a30
import base64
import cv2
import numpy as np
def base64_to_cv2(b64str):
data = base64.b64decode(b64str.encode('utf8'))
data = np.fromstring(data, np.uint8)
data = cv2.imdecode(data, cv2.IMREAD_COLOR)
return data
modules/image/Image_gan/stargan_celeba/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# stargan_celeba
|Module Name|stargan_celeba|
| :--- | :---: |
|Category|image generation|
|Network|STGAN|
|Dataset|Celeba|
|Fine-tuning supported or not|No|
|Module Size |33MB|
|Latest update date|2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/137855887-f0abca76-2735-4275-b7ad-242decf31bb3.PNG" width=600><br/>
The image attributes are: origial image, Black_Hair, Blond_Hair, Brown_Hair, Male, Aged<br/>
</p>
- ### Module Introduction
- STGAN takes the original attribute and the target attribute as input, and proposes STUs (Selective transfer units) to select and modify features of the encoder. The PaddleHub Module is trained one Celeba dataset and currently supports attributes of "Black_Hair", "Blond_Hair", "Brown_Hair", "Female", "Male", "Aged".
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.5.2
- paddlehub >= 1.0.0 | [How to install PaddleHub](../../../../docs/docs_ch/get_start/installation.rst)
- ### 2、Installation
- ```shell
$ hub install stargan_celeba==1.0.0
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```shell
$ hub run stargan_celeba --image "/PATH/TO/IMAGE" --style "target_attribute"
```
- **Parameters**
- image: image path
- style: Specify the attributes to be converted. The options are "Black_Hair", "Blond_Hair", "Brown_Hair", "Female", "Male", "Aged". You can choose one of the options.
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import paddlehub as hub
stargan = hub.Module(name="stargan_celeba")
test_img_path = ["/PATH/TO/IMAGE"]
trans_attr = ["Blond_Hair"]
# set input dict
input_dict = {"image": test_img_path, "style": trans_attr}
# execute predict and print the result
results = stargan.generate(data=input_dict)
print(results)
```
- ### 3、API
- ```python
def generate(data)
```
- Style transfer API.
- **Parameter**
- data(list[dict]): each element in the list is dict and each field is:
- image (list\[str\])： Each element in the list is the path of the image to be converted.
- style (list\[str\])： Each element in the list is a string, fill in the face attributes to be converted.
- **Return**
- res (list\[str\]): Save path of the result.
## IV. Release Note
- 1.0.0
First release
modules/image/Image_gan/stgan_celeba/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# stgan_celeba
|Module Name|stgan_celeba|
| :--- | :---: |
|Category|image generation|
|Network|STGAN|
|Dataset|Celeba|
|Fine-tuning supported or not|No|
|Module Size |287MB|
|Latest update date|2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/137856070-2a43facd-cda0-473f-8935-e61f5dd583d8.JPG" width=1200><br/>
The image attributes are: original image, Bald, Bangs, Black_Hair, Blond_Hair, Brown_Hair, Bushy_Eyebrows, Eyeglasses, Gender, Mouth_Slightly_Open, Mustache, No_Beard, Pale_Skin, Aged<br/>
</p>
- ### Module Introduction
- STGAN takes the original attribute and the target attribute as input, and proposes STUs (Selective transfer units) to select and modify features of the encoder. The PaddleHub Module is trained one Celeba dataset and currently supports attributes of "Bald", "Bangs", "Black_Hair", "Blond_Hair", "Brown_Hair", "Bushy_Eyebrows", "Eyeglasses", "Gender", "Mouth_Slightly_Open", "Mustache", "No_Beard", "Pale_Skin", "Aged".
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.5.2
- paddlehub >= 1.0.0 | [How to install PaddleHub](../../../../docs/docs_ch/get_start/installation.rst)
- ### 2、Installation
- ```shell
$ hub install stgan_celeba==1.0.0
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Command line Prediction
- ```shell
$ hub run stgan_celeba --image "/PATH/TO/IMAGE" --info "original_attributes" --style "target_attribute"
```
- **Parameters**
- image: Image path
- info: Attributes of original image, must fill in gender（ "Male" or "Female").The options are "Bald", "Bangs", "Black_Hair", "Blond_Hair", "Brown_Hair", "Bushy_Eyebrows", "Eyeglasses", "Mouth_Slightly_Open", "Mustache", "No_Beard", "Pale_Skin", "Aged". For example, the input picture is a girl with black hair, then fill in as "Female,Black_Hair".
- style: Specify the attributes to be converted. The options are "Bald", "Bangs", "Black_Hair", "Blond_Hair", "Brown_Hair", "Bushy_Eyebrows", "Eyeglasses", "Gender", "Mouth_Slightly_Open", "Mustache", "No_Beard", "Pale_Skin", "Aged". You can choose one of the options.
- If you want to call the Hub module through the command line, please refer to: [PaddleHub Command Line Instruction](../../../../docs/docs_en/tutorial/cmd_usage.rst)
- ### 2、Prediction Code Example
- ```python
import paddlehub as hub
stgan = hub.Module(name="stgan_celeba")
test_img_path = ["/PATH/TO/IMAGE"]
org_info = ["Female,Black_Hair"]
trans_attr = ["Bangs"]
# set input dict
input_dict = {"image": test_img_path, "style": trans_attr, "info": org_info}
# execute predict and print the result
results = stgan.generate(data=input_dict)
print(results)
```
- ### 3、API
- ```python
def generate(data)
```
- Style transfer API.
- **Parameter**
- data(list[dict]): Each element in the list is dict and each field is:
- image (list\[str\])： Each element in the list is the path of the image to be converted.
- style (list\[str\])： Each element in the list is a string, fill in the face attributes to be converted.
- info (list\[str\])： Represents the face attributes of the original image. Different attributes are separated by commas.
- **Return**
- res (list\[str\]): Save path of the result.
## IV. Release Note
- 1.0.0
First release
modules/image/Image_gan/style_transfer/ID_Photo_GEN/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# ID_Photo_GEN
|Module Name |ID_Photo_GEN|
| :--- | :---: |
|Category|Image generation|
|Network|HRNet_W18|
|Dataset |-|
|Fine-tuning supported or not |No|
|Module Size|28KB|
|Latest update date|2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://img-blog.csdnimg.cn/20201224163307901.jpg" >
</p>
- ### Module Introduction
- This model is based on face_landmark_localization and FCN_HRNet_W18_Face_Seg. It can generate ID photos with white, red and blue background
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 2.0.0
- paddlehub >= 2.0.0
- ### 2、Installation
- ```shell
$ hub install ID_Photo_GEN
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
model = hub.Module(name='ID_Photo_GEN')
result = model.Photo_GEN(
images=[cv2.imread('/PATH/TO/IMAGE')],
paths=None,
batch_size=1,
output_dir='output',
visualization=True,
use_gpu=False)
```
- ### 2、API
- ```python
def Photo_GEN(
images=None,
paths=None,
batch_size=1,
output_dir='output',
visualization=False,
use_gpu=False):
```
- Prediction API, generating ID photos.
- **Parameter**
* images (list[np.ndarray]): Image data, ndarray.shape is in the format [H, W, C], BGR.
* paths (list[str]): Image path
* batch_size (int): Batch size
* output_dir (str): Save path of images, output by default.
* visualization (bool): Whether to save the recognition results as picture files.
* use_gpu (bool): Use GPU or not. **set the CUDA_VISIBLE_DEVICES environment variable first if you are using GPU**
**NOTE:** Choose one of `paths` and `images` to provide input data.
- **Return**
* results (list[dict{"write":np.ndarray,"blue":np.ndarray,"red":np.ndarray}]): The list of generation results.
## IV. Release Note
- 1.0.0
First release
\ No newline at end of file
modules/image/Image_gan/style_transfer/UGATIT_83w/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# UGATIT_83w
|Module Name|UGATIT_83w|
| :--- | :---: |
|Category|Image editing|
|Network |U-GAT-IT|
|Dataset|selfie2anime|
|Fine-tuning supported or not|No|
|Module Size|41MB|
|Latest update date |2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/136651638-33cac040-edad-41ac-a9ce-7c0e678d8c52.jpg" width = "400" height = "400" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/136651644-dd1d3836-99b3-40f0-8543-37de18f9cfd9.jpg" width = "400" height = "400" hspace='10'/>
</p>
- ### Module Introduction
- UGATIT can transfer the input face image into the anime style.
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.8.2
- paddlehub >= 1.8.0
- ### 2、Installation
- ```shell
$ hub install UGATIT_83w
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
model = hub.Module(name='UGATIT_83w', use_gpu=False)
result = model.style_transfer(images=[cv2.imread('/PATH/TO/IMAGE')])
# or
# result = model.style_transfer(paths=['/PATH/TO/IMAGE'])
```
- ### 2、API
- ```python
def style_transfer(
self,
images=None,
paths=None,
batch_size=1,
output_dir='output',
visualization=False
)
```
- Style transfer API, convert the input face image into anime style.
- **Parameters**
* images (list\[numpy.ndarray\]): Image data, ndarray.shape is in the format [H, W, C], BGR.
* paths (list\[str\]): image path，default is None；
* batch\_size (int): Batch size, default is 1；
* visualization (bool): Whether to save the recognition results as picture files, default is False.
* output\_dir (str): Save path of images, `output` by default.
**NOTE:** Choose one of `paths` and `images` to provide data.
- **Return**
- res (list\[numpy.ndarray\]): Result, ndarray.shape is in the format [H, W, C].
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of Style transfer task.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m UGATIT_83w
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
# Send an HTTP request
data = {'images':[cv2_to_base64(cv2.imread("/PATH/TO/IMAGE"))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/UGATIT_83w"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# print prediction results
print(r.json()["results"])
```
## V. Release Note
- 1.0.0
First release
\ No newline at end of file
modules/image/Image_gan/style_transfer/UGATIT_92w/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# UGATIT_92w
|Module Name|UGATIT_92w|
| :--- | :---: |
|Category|Image editing|
|Network |U-GAT-IT|
|Dataset|selfie2anime|
|Fine-tuning supported or not|No|
|Module Size|41MB|
|Latest update date |2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/136651638-33cac040-edad-41ac-a9ce-7c0e678d8c52.jpg" width = "400" height = "400" hspace='10'/> <img src="https://user-images.githubusercontent.com/35907364/136653047-f00c30fb-521f-486f-8247-8d8f63649473.jpg" width = "400" height = "400" hspace='10'/>
</p>
- ### Module Introduction
- UGATIT can transfer the input face image into the anime style.
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.8.2
- paddlehub >= 1.8.0
- ### 2、Installation
- ```shell
$ hub install UGATIT_92w
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import cv2
import paddlehub as hub
model = hub.Module(name='UGATIT_92w', use_gpu=False)
result = model.style_transfer(images=[cv2.imread('/PATH/TO/IMAGE')])
# or
# result = model.style_transfer(paths=['/PATH/TO/IMAGE'])
```
- ### 2、API
- ```python
def style_transfer(
self,
images=None,
paths=None,
batch_size=1,
output_dir='output',
visualization=False
)
```
- Style transfer API, convert the input face image into anime style.
- **Parameters**
* images (list\[numpy.ndarray\]): Image data, ndarray.shape is in the format [H, W, C], BGR.
* paths (list\[str\]): Image path，default is None；
* batch\_size (int): Batch size, default is 1；
* visualization (bool): Whether to save the recognition results as picture files, default is False.
* output\_dir (str): save path of images, `output` by default.
**NOTE:** Choose one of `paths` and `images` to provide input data.
- **Return**
- res (list\[numpy.ndarray\]): Style tranfer result, ndarray.shape is in the format [H, W, C].
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of Style transfer task.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m UGATIT_92w
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
# Send an HTTP request
data = {'images':[cv2_to_base64(cv2.imread("/PATH/TO/IMAGE"))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/UGATIT_92w"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# print prediction results
print(r.json()["results"])
```
## V. Release Note
- 1.0.0
First release
\ No newline at end of file
modules/image/Image_gan/style_transfer/animegan_v2_paprika_54/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# animegan_v2_paprika_54
|Module Name |animegan_v2_paprika_54|
| :--- | :---: |
|Category |Image generation|
|Network|AnimeGAN|
|Dataset|Paprika|
|Fine-tuning supported or not|No|
|Module Size|9.4MB|
|Latest update date|2021-02-26|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://ai-studio-static-online.cdn.bcebos.com/bd002c4bb6a7427daf26988770bb18648b7d8d2bfd6746bfb9a429db4867727f" width = "450" height = "300" hspace='10'/>
<br />
Input image
<br />
<img src="https://ai-studio-static-online.cdn.bcebos.com/6574669d87b24bab9627c6e33896528b4a0bf5af1cd84ca29655d68719f2d551" width = "450" height = "300" hspace='10'/>
<br />
Output image
<br />
</p>
- ### Module Introduction
- AnimeGAN V2 image style stransfer model, the model can convert the input image into red pepper anime style, the model weight is converted from[AnimeGAN V2 official repo](https://github.com/TachibanaYoshino/AnimeGAN)。
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.8.0
- paddlehub >= 1.8.0 | [How to install PaddleHub](../../../../docs/docs_en/get_start/installation.rst)
- ### 2、Installation
- ```shell
$ hub install animegan_v2_paprika_54
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import paddlehub as hub
import cv2
model = hub.Module(name="animegan_v2_paprika_54")
result = model.style_transfer(images=[cv2.imread('/PATH/TO/IMAGE')])
# or
# result = model.style_transfer(paths=['/PATH/TO/IMAGE'])
```
- ### 2、API
- ```python
def style_transfer(images=None,
paths=None,
output_dir='output',
visualization=False,
min_size=32,
max_size=1024)
```
- Style transfer API.
- **Parameters**
- images (list\[numpy.ndarray\]): Image data, ndarray.shape is in the format [H, W, C], BGR.
- paths (list\[str\]): Image path.
- output\_dir (str): Save path of images, `output` by default.
- visualization (bool): Whether to save the results as picture files.
- min\_size (int): Minimum size, default is 32.
- max\_size (int): Maximum size, default is 1024.
**NOTE:** Choose one of `paths` and `images` to provide input data.
- **Return**
- res (list\[numpy.ndarray\]): The list of style transfer results，ndarray.shape is in the format [H, W, C].
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of style transfer.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m animegan_v2_paprika_54
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
# Send an HTTP request
data = {'images':[cv2_to_base64(cv2.imread("/PATH/TO/IMAGE"))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/animegan_v2_paprika_54"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# print prediction results
print(r.json()["results"])
```
## V. Release Note
- 1.0.0
First release.
* 1.0.1
Support paddlehub2.0.
* 1.0.2
Delete batch_size.
modules/image/Image_gan/style_transfer/animegan_v2_paprika_97/README_en.md 0 → 100644
浏览文件 @ 9f622a30
# animegan_v2_paprika_97
|Module Name |animegan_v2_paprika_97|
| :--- | :---: |
|Category |Image generation|
|Network|AnimeGAN|
|Dataset|Paprika|
|Fine-tuning supported or not|No|
|Module Size|9.7MB|
|Latest update date|2021-07-30|
|Data indicators|-|
## I. Basic Information
- ### Application Effect Display
- Sample results:
<p align="center">
<img src="https://user-images.githubusercontent.com/35907364/136652269-48b8c902-3a2b-46b7-a9f2-d500097bbb0e.jpg" width = "450" height = "300" hspace='10'/>
<br />
Input image
<br />
<img src="https://user-images.githubusercontent.com/35907364/136652280-7e9ebfd2-8a45-4b5b-b3ac-f107770525c4.jpg" width = "450" height = "300" hspace='10'/>
<br />
Output image
<br />
</p>
- ### Module Introduction
- AnimeGAN V2 image style stransfer model, the model can convert the input image into red pepper anime style, the model weight is converted from[AnimeGAN V2 official repo](https://github.com/TachibanaYoshino/AnimeGAN)。
## II. Installation
- ### 1、Environmental Dependence
- paddlepaddle >= 1.8.0
- paddlehub >= 1.8.0 | [How to install PaddleHub](../../../../docs/docs_ch/get_start/installation.rst)
- ### 2、Installation
- ```shell
$ hub install animegan_v2_paprika_97
```
- In case of any problems during installation, please refer to:[Windows_Quickstart](../../../../docs/docs_en/get_start/windows_quickstart.md)
| [Linux_Quickstart](../../../../docs/docs_en/get_start/linux_quickstart.md) | [Mac_Quickstart](../../../../docs/docs_en/get_start/mac_quickstart.md)
## III. Module API Prediction
- ### 1、Prediction Code Example
- ```python
import paddlehub as hub
import cv2
model = hub.Module(name="animegan_v2_paprika_97")
result = model.style_transfer(images=[cv2.imread('/PATH/TO/IMAGE')])
# or
# result = model.style_transfer(paths=['/PATH/TO/IMAGE'])
```
- ### 2、API
- ```python
def style_transfer(images=None,
paths=None,
output_dir='output',
visualization=False,
min_size=32,
max_size=1024)
```
- Style transfer API.
- **Parameters**
- images (list\[numpy.ndarray\]): Image data, ndarray.shape is in the format [H, W, C], BGR.
- paths (list\[str\]): Image path.
- output\_dir (str): Save path of images, `output` by default.
- visualization (bool): Whether to save the results as picture files.
- min\_size (int): Minimum size, default is 32.
- max\_size (int): Maximum size, default is 1024.
**NOTE:** Choose one of `paths` and `images` to provide input data.
- **Return**
- res (list\[numpy.ndarray\]): The list of style transfer results，ndarray.shape is in the format [H, W, C].
## IV. Server Deployment
- PaddleHub Serving can deploy an online service of style transfer.
- ### Step 1: Start PaddleHub Serving
- Run the startup command:
- ```shell
$ hub serving start -m animegan_v2_paprika_97
```
- The servitization API is now deployed and the default port number is 8866.
- **NOTE:** If GPU is used for prediction, set CUDA_VISIBLE_DEVICES environment variable before the service, otherwise it need not be set.
- ### Step 2: Send a predictive request
- With a configured server, use the following lines of code to send the prediction request and obtain the result
- ```python
import requests
import json
import cv2
import base64
def cv2_to_base64(image):
data = cv2.imencode('.jpg', image)[1]
return base64.b64encode(data.tostring()).decode('utf8')
# Send an HTTP request
data = {'images':[cv2_to_base64(cv2.imread("/PATH/TO/IMAGE"))]}
headers = {"Content-type": "application/json"}
url = "http://127.0.0.1:8866/predict/animegan_v2_paprika_97"
r = requests.post(url=url, headers=headers, data=json.dumps(data))
# print prediction results
print(r.json()["results"])
```
## V. Release Note
- 1.0.0
First release.
* 1.0.1
Support paddlehub2.0.
* 1.0.2
Delete batch_size.
modules/image/Image_gan/style_transfer/msgnet/README.md
浏览文件 @ 9f622a30
......@@ -50,13 +50,14 @@ $ hub run msgnet --input_path "/PATH/TO/ORIGIN/IMAGE" --style_path "/PATH/TO/STY
- ### 2.预测代码示例
```python
import paddle
import paddlehub as hub
if __name__ == '__main__':
model = hub.Module(name='msgnet')
result = model.predict(origin=["venice-boat.jpg"], style="candy.jpg", visualization=True, save_path ='style_tranfer')
result = model.predict(origin=["/PATH/TO/ORIGIN/IMAGE"], style="/PATH/TO/STYLE/IMAGE", visualization=True, save_path ="/PATH/TO/SAVE/IMAGE")
```
......@@ -86,7 +87,7 @@ if __name__ == '__main__':
- `transforms`: 数据预处理方式。
- `mode`: 选择数据模式，可选项有 `train`, `test`， 默认为`train`。
- 数据集的准备代码可以参考 [minicoco.py](../../paddlehub/datasets/flowers.py)。`hub.datasets.MiniCOCO()`会自动从网络下载数据集并解压到用户目录下`$HOME/.paddlehub/dataset`目录。
- 数据集的准备代码可以参考 [minicoco.py](../../paddlehub/datasets/minicoco.py)。`hub.datasets.MiniCOCO()`会自动从网络下载数据集并解压到用户目录下`$HOME/.paddlehub/dataset`目录。
- Step3: 加载预训练模型
......@@ -117,7 +118,7 @@ if __name__ == '__main__':
if __name__ == '__main__':
model = hub.Module(name='msgnet', load_checkpoint="/PATH/TO/CHECKPOINT")
result = model.predict(origin=["venice-boat.jpg"], style="candy.jpg", visualization=True, save_path ='style_tranfer')
result = model.predict(origin=["/PATH/TO/ORIGIN/IMAGE"], style="/PATH/TO/STYLE/IMAGE", visualization=True, save_path ="/PATH/TO/SAVE/IMAGE")
```
- 参数配置正确后，请执行脚本`python predict.py`， 加载模型具体可参见[加载](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0-rc/api/paddle/framework/io/load_cn.html#load)。
......
modules/image/classification/resnet50_vd_animals/README.md
浏览文件 @ 9f622a30
......@@ -44,7 +44,7 @@
hub run resnet50_vd_animals --input_path "/PATH/TO/IMAGE"
```
- ### 2、代码示例
- ### 2、预测代码示例
- ```python
import paddlehub as hub
......
modules/image/classification/resnet50_vd_imagenet_ssld/README.md
浏览文件 @ 9f622a30
......@@ -50,7 +50,7 @@
if __name__ == '__main__':
model = hub.Module(name='resnet50_vd_imagenet_ssld')
result = model.predict(['flower.jpg'])
result = model.predict(['/PATH/TO/IMAGE'])
```
- ### 3.如何开始Fine-tune
......@@ -134,7 +134,7 @@
if __name__ == '__main__':
model = hub.Module(name='resnet50_vd_imagenet_ssld', label_list=["roses", "tulips", "daisy", "sunflowers", "dandelion"], load_checkpoint='/PATH/TO/CHECKPOINT')
result = model.predict(['flower.jpg'])
result = model.predict(['/PATH/TO/IMAGE'])
```
......
modules/image/semantic_segmentation/ExtremeC3_Portrait_Segmentation/README.md
浏览文件 @ 9f622a30
......@@ -44,7 +44,7 @@
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
import cv2
......@@ -60,26 +60,26 @@
visualization=False)
```
- ### 2、API
- ### 2、API
```python
def Segmentation(
```python
def Segmentation(
images=None,
paths=None,
batch_size=1,
output_dir='output',
visualization=False):
```
- 人像分割 API
```
- 人像分割 API
- **参数**
- **参数**
* images (list[np.ndarray]) : 输入图像数据列表（BGR）
* paths (list[str]) : 输入图像路径列表
* batch_size (int) : 数据批大小
* output_dir (str) : 可视化图像输出目录
* visualization (bool) : 是否可视化
- **返回**
- **返回**
* results (list[dict{"mask":np.ndarray,"result":np.ndarray}]): 输出图像数据列表
## 四、更新历史
......
modules/image/semantic_segmentation/U2Net/README.md
浏览文件 @ 9f622a30
......@@ -43,7 +43,7 @@
| [零基础Linux安装](../../../../docs/docs_ch/get_start/linux_quickstart.md) | [零基础MacOS安装](../../../../docs/docs_ch/get_start/mac_quickstart.md)
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
import cv2
......
modules/image/semantic_segmentation/U2Net/README_en.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/image/semantic_segmentation/U2Netp/README.md
浏览文件 @ 9f622a30
......@@ -47,7 +47,7 @@
| [零基础Linux安装](../../../../docs/docs_ch/get_start/linux_quickstart.md) | [零基础MacOS安装](../../../../docs/docs_ch/get_start/mac_quickstart.md)
## 三、模型API预测
- ### 1、代码示例
- ### 1、预测代码示例
```python
import cv2
......
modules/image/semantic_segmentation/U2Netp/README_en.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/image/semantic_segmentation/ace2p/README_en.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/text/text_generation/ernie_gen/README_en.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/text/text_to_knowledge/nptag/README.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/text/text_to_knowledge/nptag/module.py 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/video/Video_editing/SkyAR/README_en.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/video/classification/tsm_kinetics400/README.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
modules/video/classification/tsn_kinetics400/README.md 0 → 100644
浏览文件 @ 9f622a30
此差异已折叠。
paddlehub/__init__.py
浏览文件 @ 9f622a30
此差异已折叠。
paddlehub/server/server.py
浏览文件 @ 9f622a30
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或