diff --git a/README.md b/README.md index f4e16b4fc9f0e33a01d6664600b77b0ce563317a..e72558f5c800842f520c8d437bb2a6c9388eb150 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,83 @@ -English | [简体中文](./README.md) +English | [简体中文](./README_cn.md) # PaddleGAN -still under development!! +PaddleGAN is an development kit of Generative Adversarial Network based on PaddlePaddle. -## Download Dataset -This script could download several dataset for paired images for image2image translation task. +### Image Translation +![](./docs/imgs/A2B.png) +![](./docs/imgs/B2A.png) -``` -cd PaddleGAN/script/ -bash pix2pix_download.sh [cityscapes|facades|edges2handbags|edges2shoes|maps] -``` -## Train -``` -python -u tools/main.py --config-file configs/cyclegan-cityscapes.yaml -``` +### Makeup shifter +![](./docs/imgs/makeup_shifter.png) + +### Old video restore +![](./docs/imgs/color_sr_peking.gif) + +### Super resolution + +![](./docs/imgs/sr_demo.png) + + +Features: + +- Highly Flexible: + + Components are designed to be modular. Model architectures, as well as data +preprocess pipelines, can be easily customized with simple configuration +changes. + +- Rich applications: + + PaddleGAN provides rich of applications, such as image generation, image restore, image colorization, video interpolate, makeup shifter. + +## Install + +### 1. install paddlepaddle + +PaddleGAN work with: +* PaddlePaddle >= 2.0.0-rc +* Python >= 3.5+ -continue train from last checkpoint ``` -python -u tools/main.py --config-file configs/cyclegan-cityscapes.yaml --resume your_checkpoint_path +pip install -U paddlepaddle-gpu ``` -multiple gpus train: +### 2. install ppgan + ``` -CUDA_VISIBLE_DEVICES=0,1 python -m paddle.distributed.launch tools/main.py --config-file configs/pix2pix-cityscapes.yaml +python -m pip install 'git+https://github.com/PaddlePaddle/PaddleGAN.git' ``` -## Evaluate +Or install it from a local clone ``` -python tools/main.py --config-file configs/cyclegan-cityscapes.yaml --evaluate-only --load your_weight_path +git clone https://github.com/PaddlePaddle/PaddleGAN +cd PaddleGAN + +pip install -v -e . # or "python setup.py develop" ``` + +## Data Prepare +Please refer to [data prepare](./docs/data_prepare.md) for dataset preparation. + +## Get Start +Please refer [get stated](./docs/get_started.md) for the basic usage of PaddleGAN. + +## Model tutorial +* [Pixel2Pixel and CycleGAN](./docs/tutorials/pix2pix_cyclegan.md) +* [PSGAN](./docs/tutorials/psgan.md) +* [Video restore](./docs/tutorails/video_restore.md) +* [Motion driving](./docs/tutorials/motion_driving.md) + +## License +PaddleGAN is released under the [Apache 2.0 license](LICENSE). + +## Contributing + +Contributions are highly welcomed and we would really appreciate your feedback!! + +## External Projects + +External gan projects in the community that base on PaddlePaddle: + ++ [PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN) diff --git a/README_cn.md b/README_cn.md new file mode 100644 index 0000000000000000000000000000000000000000..7b213833a31666865d505802f35e4a03e15366f3 --- /dev/null +++ b/README_cn.md @@ -0,0 +1,81 @@ +[English](./README.md) | 简体中文 + +# PaddleGAN + +PaddleGAN 是一个基于飞桨的生成对抗网络开发工具包. + +### 图片变换 +![](./docs/imgs/A2B.png) +![](./docs/imgs/B2A.png) + +### 妆容迁移 +![](./docs/imgs/makeup_shifter.png) + +### 老视频修复 +![](./docs/imgs/color_sr_peking.gif) + +### 超分辨率 +![](./docs/imgs/sr_demo.png) + + +特性: + +- 高度的灵活性: + + 模块化设计，解耦各个网络组件，开发者轻松搭建、试用各种检测模型及优化策略，快速得到高性能、定制化的算法。 + +- 丰富的应用: + + PaddleGAN 提供了非常多的应用，比如说图像生成，图像修复，图像上色，视频补帧，人脸妆容迁移等. + +## 安装 + +### 1. 安装 paddlepaddle + +PaddleGAN 所需的版本: +* PaddlePaddle >= 2.0.0-rc +* Python >= 3.5+ + +``` +pip install -U paddlepaddle-gpu +``` + +### 2. 安装ppgan +``` +python -m pip install 'git+https://github.com/PaddlePaddle/PaddleGAN.git' +``` + +或者通过将项目克隆到本地 +``` +git clone https://github.com/PaddlePaddle/PaddleGAN +cd PaddleGAN + +pip install -v -e . # or "python setup.py develop" +``` + +## 数据准备 +请参考 [数据准备](./docs/data_prepare.md) 来准备对应的数据. + + +## 快速开始 +训练，预测，推理等请参考 [快速开始](./docs/get_started.md). + +## 模型教程 +* [Pixel2Pixel and CycleGAN](./docs/tutorals/pix2pix_cyclegan.md) +* [PSGAN](./docs/tutorals/psgan.md) +* [视频修复](./docs/tutorails/video_restore.md) +* [动作驱动](./docs/tutorials/motion_driving.md) + +## 许可证书 +本项目的发布受[Apache 2.0 license](LICENSE)许可认证。 + + +## 贡献代码 + +我们非常欢迎你可以为PaddleGAN提供代码，也十分感谢你的反馈。 + +## 外部项目 + +外部基于飞桨的生成对抗网络模型 + ++ [PaddleGAN](https://github.com/PaddlePaddle/PaddleGAN) diff --git a/applications/tools/video-enhance.py b/applications/tools/video-enhance.py index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..40f066df814993b9544dabf07652fbb073a42022 100644 --- a/applications/tools/video-enhance.py +++ b/applications/tools/video-enhance.py @@ -0,0 +1,119 @@ +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve. +# +#Licensed under the Apache License, Version 2.0 (the "License"); +#you may not use this file except in compliance with the License. +#You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +#Unless required by applicable law or agreed to in writing, software +#distributed under the License is distributed on an "AS IS" BASIS, +#WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +#See the License for the specific language governing permissions and +#limitations under the License. + +import argparse +import paddle + +from ppgan.apps import DAINPredictor +from ppgan.apps import DeepRemasterPredictor +from ppgan.apps import DeOldifyPredictor +from ppgan.apps import RealSRPredictor +from ppgan.apps import EDVRPredictor + +parser = argparse.ArgumentParser(description='Fix video') +parser.add_argument('--input', type=str, default=None, help='Input video') +parser.add_argument('--output', type=str, default='output', help='output dir') +parser.add_argument('--DAIN_weight', + type=str, + default=None, + help='Path to model weight') +parser.add_argument('--DeepRemaster_weight', + type=str, + default=None, + help='Path to model weight') +parser.add_argument('--DeOldify_weight', + type=str, + default=None, + help='Path to model weight') +parser.add_argument('--RealSR_weight', + type=str, + default=None, + help='Path to model weight') +parser.add_argument('--EDVR_weight', + type=str, + default=None, + help='Path to model weight') +# DAIN args +parser.add_argument('--time_step', + type=float, + default=0.5, + help='choose the time steps') +# DeepRemaster args +parser.add_argument('--reference_dir', + type=str, + default=None, + help='Path to the reference image directory') +parser.add_argument('--colorization', + action='store_true', + default=False, + help='Remaster with colorization') +parser.add_argument('--mindim', + type=int, + default=360, + help='Length of minimum image edges') +# DeOldify args +parser.add_argument('--render_factor', + type=int, + default=32, + help='model inputsize=render_factor*16') +#process order support model name:[DAIN, DeepRemaster, DeOldify, RealSR, EDVR] +parser.add_argument('--proccess_order', + type=str, + default='none', + nargs='+', + help='Process order') + +if __name__ == "__main__": + args = parser.parse_args() + + orders = args.proccess_order + temp_video_path = None + + for order in orders: + print('Model {} proccess start..'.format(order)) + if temp_video_path is None: + temp_video_path = args.input + if order == 'DAIN': + paddle.enable_static() + predictor = DAINPredictor(args.output, + weight_path=args.DAIN_weight, + time_step=args.time_step) + frames_path, temp_video_path = predictor.run(temp_video_path) + paddle.disable_static() + elif order == 'DeepRemaster': + predictor = DeepRemasterPredictor( + args.output, + weight_path=args.DeepRemaster_weight, + colorization=args.colorization, + reference_dir=args.reference_dir, + mindim=args.mindim) + frames_path, temp_video_path = predictor.run(temp_video_path) + elif order == 'DeOldify': + predictor = DeOldifyPredictor(args.output, + weight_path=args.DeOldify_weight, + render_factor=args.render_factor) + frames_path, temp_video_path = predictor.run(temp_video_path) + elif order == 'RealSR': + predictor = RealSRPredictor(args.output, + weight_path=args.RealSR_weight) + frames_path, temp_video_path = predictor.run(temp_video_path) + elif order == 'EDVR': + paddle.enable_static() + predictor = EDVRPredictor(args.output, weight_path=args.EDVR_weight) + frames_path, temp_video_path = predictor.run(temp_video_path) + paddle.disable_static() + + print('Model {} output frames path:'.format(order), frames_path) + print('Model {} output video path:'.format(order), temp_video_path) + print('Model {} proccess done!'.format(order)) diff --git a/docs/data_prepare.md b/docs/data_prepare.md new file mode 100644 index 0000000000000000000000000000000000000000..6c25aa214c4e4d79178546866e952bc1f7ccc97f --- /dev/null +++ b/docs/data_prepare.md @@ -0,0 +1,35 @@ +## data prepare + +It is recommended to symlink the dataset root to `$PaddleGAN/data`. + +``` +PaddleGAN +|-- configs +|-- data +| |-- cityscapes +| | ├── test +| | ├── testA +| | ├── testB +| | ├── train +| | ├── trainA +| | └── trainB +| ├── horse2zebra +| | ├── testA +| | ├── testB +| | ├── trainA +| | └── trainB +| └── facades +| ├── test +| ├── train +| └── val +|-- docs +|-- ppgan +|-- tools + +``` + +### cyclegan datasets +more dataset for cyclegan you can download from [here](https://people.eecs.berkeley.edu/~taesung_park/CycleGAN/datasets/) + +### pix2pix datasets +more dataset for pix2pix you can download from [here](hhttps://people.eecs.berkeley.edu/~tinghuiz/projects/pix2pix/datasets/) diff --git a/docs/get_started.md b/docs/get_started.md new file mode 100644 index 0000000000000000000000000000000000000000..024e8dec2bfac2afc51d472fa8617c0db82986fe --- /dev/null +++ b/docs/get_started.md @@ -0,0 +1,22 @@ + +## Getting started with PaddleGAN + +### Train +``` +python -u tools/main.py --config-file configs/cyclegan_cityscapes.yaml +``` + +continue train from last checkpoint +``` +python -u tools/main.py --config-file configs/cyclegan_cityscapes.yaml --resume your_checkpoint_path +``` + +multiple gpus train: +``` +CUDA_VISIBLE_DEVICES=0,1 python -m paddle.distributed.launch tools/main.py --config-file configs/pix2pix_cityscapes.yaml +``` + +### Evaluate +``` +python tools/main.py --config-file configs/cyclegan_cityscapes.yaml --evaluate-only --load your_weight_path +``` diff --git a/docs/imgs/A2B.png b/docs/imgs/A2B.png new file mode 100644 index 0000000000000000000000000000000000000000..b67466da9bdf04344ac6a8f417169414641be664 Binary files /dev/null and b/docs/imgs/A2B.png differ diff --git a/docs/imgs/B2A.png b/docs/imgs/B2A.png new file mode 100644 index 0000000000000000000000000000000000000000..116b59e3be7b26fc6ae9c2457b4f7a59d635c562 Binary files /dev/null and b/docs/imgs/B2A.png differ diff --git a/docs/imgs/color_sr_peking.gif b/docs/imgs/color_sr_peking.gif new file mode 100644 index 0000000000000000000000000000000000000000..9792390cf4fcc2677c248e1d01ee7c1a262a553e Binary files /dev/null and b/docs/imgs/color_sr_peking.gif differ diff --git a/docs/imgs/makeup_shifter.png b/docs/imgs/makeup_shifter.png new file mode 100644 index 0000000000000000000000000000000000000000..63cb5626de74175e74f254d2fdeb951c6afc9e8e Binary files /dev/null and b/docs/imgs/makeup_shifter.png differ diff --git a/docs/imgs/sr_demo.png b/docs/imgs/sr_demo.png new file mode 100644 index 0000000000000000000000000000000000000000..885ddb308bdf64a481122ec6ac3bca614a0dfa1f Binary files /dev/null and b/docs/imgs/sr_demo.png differ diff --git a/docs/tutorials/motion_driving.md b/docs/tutorials/motion_driving.md new file mode 100644 index 0000000000000000000000000000000000000000..92ca73667e885b7fd0f4eeb4364d43facc8a698a --- /dev/null +++ b/docs/tutorials/motion_driving.md @@ -0,0 +1 @@ +## to be added diff --git a/docs/tutorials/pix2pix_cyclegan.md b/docs/tutorials/pix2pix_cyclegan.md new file mode 100644 index 0000000000000000000000000000000000000000..92ca73667e885b7fd0f4eeb4364d43facc8a698a --- /dev/null +++ b/docs/tutorials/pix2pix_cyclegan.md @@ -0,0 +1 @@ +## to be added diff --git a/docs/tutorials/psgan.md b/docs/tutorials/psgan.md new file mode 100644 index 0000000000000000000000000000000000000000..92ca73667e885b7fd0f4eeb4364d43facc8a698a --- /dev/null +++ b/docs/tutorials/psgan.md @@ -0,0 +1 @@ +## to be added diff --git a/docs/tutorials/video_restore.md b/docs/tutorials/video_restore.md new file mode 100644 index 0000000000000000000000000000000000000000..92ca73667e885b7fd0f4eeb4364d43facc8a698a --- /dev/null +++ b/docs/tutorials/video_restore.md @@ -0,0 +1 @@ +## to be added