From 1e3b82e1ddb475dc5a0f21d20d5bf61b9d5ea657 Mon Sep 17 00:00:00 2001 From: wangna11BD <79366697+wangna11BD@users.noreply.github.com> Date: Tue, 21 Dec 2021 17:00:12 +0800 Subject: [PATCH] add docs (#536) --- README.md | 4 +++ README_cn.md | 14 +++++--- docs/en_US/tutorials/face_enhancement.md | 41 ++++++++++++++++++++++++ docs/zh_CN/tutorials/face_enhancement.md | 41 ++++++++++++++++++++++++ 4 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 docs/en_US/tutorials/face_enhancement.md create mode 100644 docs/zh_CN/tutorials/face_enhancement.md diff --git a/README.md b/README.md index 91c8cea..dc11e65 100644 --- a/README.md +++ b/README.md @@ -103,10 +103,14 @@ GAN-Generative Adversarial Network, was praised by "the Father of Convolutional * [Photo2Cartoon](./docs/en_US/tutorials/photo2cartoon.md) * [Wav2Lip](./docs/en_US/tutorials/wav2lip.md) * [Single Image Super Resolution(SISR)](./docs/en_US/tutorials/single_image_super_resolution.md) + * Including: RealSR, ESRGAN, LESRCNN, PAN, DRN * [Video Super Resolution(VSR)](./docs/en_US/tutorials/video_super_resolution.md) + * Including: ⭐ PP-MSVSR ⭐, EDVR, BasicVSR, BasicVSR++ * [StyleGAN2](./docs/en_US/tutorials/styleganv2.md) * [Pixel2Style2Pixel](./docs/en_US/tutorials/pixel2style2pixel.md) * [StarGANv2](docs/en_US/tutorials/starganv2.md) +* [MPR Net](./docs/en_US/tutorials/mpr_net.md) +* [FaceEnhancement](./docs/en_US/tutorials/face_enhancement.md) ## Composite Application diff --git a/README_cn.md b/README_cn.md index 4c332b4..b5c92cc 100644 --- a/README_cn.md +++ b/README_cn.md @@ -125,16 +125,22 @@ GAN--生成对抗网络，被“卷积网络之父”**Yann LeCun（杨立昆） * 动作迁移 * 人脸表情迁移：[First Order Motion Model](./docs/zh_CN/tutorials/motion_driving.md) * 唇形合成：[Wav2Lip](docs/zh_CN/tutorials/wav2lip.md) -* 生成 +* 基础GAN * [DCGAN](https://github.com/PaddlePaddle/PaddleGAN/blob/develop/ppgan/models/dc_gan_model.py) * WGAN +* 人脸生成 * 人脸生成：[StyleGAN2](./docs/zh_CN/tutorials/styleganv2.md) - * 图像编码：[Pixel2Style2Pixel](./docs/zh_CN/tutorials/pixel2style2pixel.md) + * 人脸编码：[Pixel2Style2Pixel](./docs/zh_CN/tutorials/pixel2style2pixel.md) + * 人脸增强：[FaceEnhancement](./docs/zh_CN/tutorials/face_enhancement.md) + * 人脸解析：[FaceParsing](./docs/zh_CN/tutorials/face_parse.md) * 分辨率提升 * 单张图片超分：[Single Image Super Resolution(SISR)](./docs/zh_CN/tutorials/single_image_super_resolution.md) + * 包含模型：RealSR、ESRGAN、LESRCNN、PAN、DRN * 视频超分：[Video Super Resolution(VSR)](./docs/zh_CN/tutorials/video_super_resolution.md) -* 语义分割 - * 人脸解析：[FaceParsing](./docs/zh_CN/tutorials/face_parse.md) + * 包含模型：⭐ PP-MSVSR ⭐、EDVR、BasicVSR、BasicVSR++ +* 图像视频修复 + * 图像去模糊去噪去雨：[MPR Net](./docs/zh_CN/tutorials/mpr_net.md) + * 视频去模糊：[EDVR](./docs/zh_CN/tutorials/video_super_resolution.md) ## 产业级应用 diff --git a/docs/en_US/tutorials/face_enhancement.md b/docs/en_US/tutorials/face_enhancement.md new file mode 100644 index 0000000..2c3b87b --- /dev/null +++ b/docs/en_US/tutorials/face_enhancement.md @@ -0,0 +1,41 @@ +# Face Enhancement + +## 1. face enhancement introduction + +Blind face restoration (BFR) from severely degraded face images in the wild is a very challenging problem. Due to the high illness of the problem and the complex unknown degradation, directly training a deep neural network (DNN) usually cannot lead to acceptable results. Existing generative adversarial network (GAN) based methods can produce better results but tend to generate over-smoothed restorations. Here we provide the [GPEN](https://arxiv.org/abs/2105.06070) model. GPEN was proposed by first learning a GAN for high-quality face image generation and embedding it into a U-shaped DNN as a prior decoder, then fine-tuning the GAN prior embedded DNN with a set of synthesized low-quality face images. The GAN blocks are designed to ensure that the latent code and noise input to the GAN can be respectively generated from the deep and shallow features of the DNN, controlling the global face structure, local face details and background of the reconstructed image. The proposed GAN prior embedded network (GPEN) is easy-to-implement, and it can generate visually photo-realistic results. Experiments demonstrated that the proposed GPEN achieves significantly superior results to state-of-the-art BFR methods both quantitatively and qualitatively, especially for the restoration of severely degraded face images in the wild. + +## How to use + +### face enhancement + +The user could use the following command to do face enhancement and select the local image as input： + +```python +import paddle +from ppgan.faceutils.face_enhancement import FaceEnhancement + +faceenhancer = FaceEnhancement() +img = faceenhancer.enhance_from_image(img) +``` + +### Train (TODO) + +In the future, training scripts will be added to facilitate users to train more types of GPEN. + +## Results + +![1](https://user-images.githubusercontent.com/79366697/146891109-d204497f-7e71-4899-bc65-e1b101ce6293.jpg) + +## Reference + +``` +@inproceedings{inproceedings, +author = {Yang, Tao and Ren, Peiran and Xie, Xuansong and Zhang, Lei}, +year = {2021}, +month = {06}, +pages = {672-681}, +title = {GAN Prior Embedded Network for Blind Face Restoration in the Wild}, +doi = {10.1109/CVPR46437.2021.00073} +} + +``` diff --git a/docs/zh_CN/tutorials/face_enhancement.md b/docs/zh_CN/tutorials/face_enhancement.md new file mode 100644 index 0000000..3bb9ace --- /dev/null +++ b/docs/zh_CN/tutorials/face_enhancement.md @@ -0,0 +1,41 @@ +# 人脸增强 + +## 1. 人脸增强简介 + +从严重退化的人脸图像中恢复出人脸是一个非常具有挑战性的问题。由于问题的严重性和复杂的未知退化，直接训练深度神经网络通常无法得到可接受的结果。现有的基于生成对抗网络 (GAN) 的方法可以产生更好的结果，但往往会产生过度平滑的恢复。这里我们提供[GPEN](https://arxiv.org/abs/2105.06070)模型来进行人脸增强。GPEN模型首先学习用于生成高质量人脸图像的GAN并将其嵌入到U形DNN作为先验解码器，然后使用一组合成的低质量人脸图像对GAN先验嵌入DNN进行微调。 GAN 模块的设计是为了确保输入到 GAN 的隐码和噪声可以分别从 DNN 的深层和浅层特征中生成，控制重建图像的全局人脸结构、局部人脸细节和背景。所提出的 GAN 先验嵌入网络 (GPEN) 易于实现，并且可以生成视觉上逼真的结果。实验表明，GPEN 在数量和质量上都比最先进的 BFR 方法取得了显着优越的结果，特别是对于野外严重退化的人脸图像的恢复。 + +## 使用方法 + +### 人脸增强 + +用户使用如下代码进行人脸增强，选择本地图像作为输入： + +```python +import paddle +from ppgan.faceutils.face_enhancement import FaceEnhancement + +faceenhancer = FaceEnhancement() +img = faceenhancer.enhance_from_image(img) +``` + +### 训练(TODO) + +未来还将添加训练脚本方便用户训练出更多类型的 GPEN 人脸增强。 + +## 人脸增强结果展示 + +![1](https://user-images.githubusercontent.com/79366697/146891109-d204497f-7e71-4899-bc65-e1b101ce6293.jpg) + +## 参考文献 + +``` +@inproceedings{inproceedings, +author = {Yang, Tao and Ren, Peiran and Xie, Xuansong and Zhang, Lei}, +year = {2021}, +month = {06}, +pages = {672-681}, +title = {GAN Prior Embedded Network for Blind Face Restoration in the Wild}, +doi = {10.1109/CVPR46437.2021.00073} +} + +``` -- GitLab