diff --git a/README.md b/README.md index eee5c49d3b17027efc96dd45548c4b550d99c866..d9ff2c7f80536840344ca44534bb62209f3b3f07 100644 --- a/README.md +++ b/README.md @@ -15,24 +15,6 @@ GAN-Generative Adversarial Network, was praised by "the Father of Convolutional ## 🎪 Hot Activities -- 🔥 **2021.7.9-2021.9** 🔥 - - **💙 AI Creation Camp 💙** - - **You can implement any abilities in PaddleGAN with Wechaty to create your own chat robot 🤖 !** - - **A plenty of gifts 🎁 waiting for you!** - - **💰First Prize: 30,000RMB** - - **🎮 Second Prize: PS5** - - **🕶 Third Prize: VR Glass** - - **🏵 Most Popular Prize: 3D Printer** - - **Still hezitating? Click here and sign up!** **https://aistudio.baidu.com/aistudio/competition/detail/98** - - 2021.4.15~4.22 GAN 7 Days Course Camp: Baidu Senior Research Developers help you learn the basic and advanced GAN knowledge in 7 days! @@ -41,6 +23,11 @@ GAN-Generative Adversarial Network, was praised by "the Father of Convolutional ## 🚀 Recent Updates +- 🔥 **Latest Release: [PP-MSVSR](./docs/en_US/tutorials/video_super_resolution.md)** 🔥 + - **Video Super Resolution SOTA models** +
+ +
- 😍 **Boy or Girl？：[StyleGAN V2 Face Editing](./docs/en_US/tutorials/styleganv2editing.md)-Changing genders！** 😍 - **[Online Toturials](https://aistudio.baidu.com/aistudio/projectdetail/2565277?contributionType=1)**
diff --git a/docs/en_US/tutorials/video_super_resolution.md b/docs/en_US/tutorials/video_super_resolution.md index 380015da4300dc354ce5c053a04707d01595bb77..3c42445972588b940565453b7c11ba82c3930627 100644 --- a/docs/en_US/tutorials/video_super_resolution.md +++ b/docs/en_US/tutorials/video_super_resolution.md @@ -5,13 +5,19 @@ Video super-resolution originates from image super-resolution, which aims to recover high-resolution (HR) images from one or more low resolution (LR) images. The difference between them is that the video is composed of multiple frames, so the video super-resolution usually uses the information between frames to repair. Here we provide the video super-resolution model [EDVR](https://arxiv.org/pdf/1905.02716.pdf), [BasicVSR](https://arxiv.org/pdf/2012.02181.pdf),[IconVSR](https://arxiv.org/pdf/2012.02181.pdf),[BasicVSR++](https://arxiv.org/pdf/2104.13371v1.pdf), and PP-MSVSR. +### 🔥 PP-MSVSR 🔥 + [PP-MSVSR](https://arxiv.org/pdf/2112.02828.pdf) is a multi-stage VSR deep architecture, with local fusion module, auxiliary loss and refined align module to refine the enhanced result progressively. Specifically, in order to strengthen the fusion of features across frames in feature propagation, a local fusion module is designed in stage-1 to perform local feature fusion before feature propagation. Moreover, an auxiliary loss in stage-2 is introduced to make the features obtained by the propagation module reserve more correlated information connected to the HR space, and introduced a refined align module in stage-3 to make full use of the feature information of the previous stage. Extensive experiments substantiate that PP-MSVSR achieves a promising performance of Vid4 datasets, which PSNR metric can achieve 28.13 with only 1.45M parameters. + + Additionally, [PP-MSVSR](https://arxiv.org/pdf/2112.02828.pdf) provides two different models with 1.45M and 7.4M parameters in order to satisfy different requirements. + +### EDVR [EDVR](https://arxiv.org/pdf/1905.02716.pdf) wins the champions and outperforms the second place by a large margin in all four tracks in the NTIRE19 video restoration and enhancement challenges. The main difficulties of video super-resolution from two aspects: (1) how to align multiple frames given large motions, and (2) how to effectively fuse different frames with diverse motion and blur. First, to handle large motions, EDVR devise a Pyramid, Cascading and Deformable (PCD) alignment module, in which frame alignment is done at the feature level using deformable convolutions in a coarse-to-fine manner. Second, EDVR propose a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. [BasicVSR](https://arxiv.org/pdf/2012.02181.pdf) reconsiders some most essential components for VSR guided by four basic functionalities, i.e., Propagation, Alignment, Aggregation, and Upsampling. By reusing some existing components added with minimal redesigns, a succinct pipeline, BasicVSR, achieves appealing improvements in terms of speed and restoration quality in comparison to many state-of-the-art algorithms. By presenting an informationrefill mechanism and a coupled propagation scheme to facilitate information aggregation, the BasicVSR can be expanded to [IconVSR](https://arxiv.org/pdf/2012.02181.pdf), which can serve as strong baselines for future VSR approaches. [BasicVSR++](https://arxiv.org/pdf/2104.13371v1.pdf) redesign BasicVSR by proposing second-order grid propagation and flowguided deformable alignment. By empowering the recurrent framework with the enhanced propagation and alignment, BasicVSR++ can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a similar computational constraint. In particular, BasicVSR++ surpasses BasicVSR by 0.82 dB in PSNR with similar number of parameters. In NTIRE 2021, BasicVSR++ obtains three champions and one runner-up in the Video Super-Resolution and Compressed Video Enhancement Challenges. - PP-MSVSR is a multi-stage VSR deep architecture, with local fusion module, auxiliary loss and refined align module to refine the enhanced result progressively. Specifically, in order to strengthen the fusion of features across frames in feature propagation, a local fusion module is designed in stage-1 to perform local feature fusion before feature propagation. Moreover, an auxiliary loss in stage-2 is introduced to make the features obtained by the propagation module reserve more correlated information connected to the HR space, and introduced a refined align module in stage-3 to make full use of the feature information of the previous stage. Extensive experiments substantiate that PP-MSVSR achieves a promising performance of Vid4 datasets, which PSNR metric can achieve 28.13 with only 1.45M parameters. + @@ -202,6 +208,10 @@ VSR quantitative comparis on the Vimeo90K, Vid4, UDM10 - 4. [PP-MSVSR: Multi-Stage Video Super-Resolution]() ``` - @article{ + @article{jiang2021PP-MSVSR, + author = {Jiang, Lielin and Wang, Na and Dang, Qingqing and Liu, Rui and Lai, Baohua}, + title = {PP-MSVSR: Multi-Stage Video Super-Resolution}, + booktitle = {arXiv preprint arXiv:2112.02828}, + year = {2021} } ``` diff --git a/docs/zh_CN/tutorials/video_super_resolution.md b/docs/zh_CN/tutorials/video_super_resolution.md index 673d3a4e8f8f015d9620221d42370819644516d2..706e73b81bbbdc9b9339a0678dde5cfec96631f2 100644 --- a/docs/zh_CN/tutorials/video_super_resolution.md +++ b/docs/zh_CN/tutorials/video_super_resolution.md @@ -8,9 +8,9 @@ 这里我们提供百度自研SOTA超分系列模型PP-MSVSR、业界领先视频超分模型[EDVR](https://arxiv.org/pdf/1905.02716.pdf)、[BasicVSR](https://arxiv.org/pdf/2012.02181.pdf)，[IconVSR](https://arxiv.org/pdf/2012.02181.pdf)和[BasicVSR++](https://arxiv.org/pdf/2104.13371v1.pdf)。 ### ⭐ PP-MSVSR ⭐ - 百度自研的PP-MSVSR是一种多阶段视频超分深度架构，具有局部融合模块、辅助损失和细化对齐模块，以逐步细化增强结果。具体来说，在第一阶段设计了局部融合模块，在特征传播之前进行局部特征融合, 以加强特征传播中跨帧特征的融合。在第二阶段中引入了一个辅助损失，使传播模块获得的特征保留了更多与HR空间相关的信息。在第三阶段中引入了一个细化的对齐模块，以充分利用前一阶段传播模块的特征信息。大量实验证实，PP-MSVSR在Vid4数据集性能优异，仅使用 1.45M 参数PSNR指标即可达到28.13dB。 + 百度自研的[PP-MSVSR](https://arxiv.org/pdf/2112.02828.pdf)是一种多阶段视频超分深度架构，具有局部融合模块、辅助损失和细化对齐模块，以逐步细化增强结果。具体来说，在第一阶段设计了局部融合模块，在特征传播之前进行局部特征融合, 以加强特征传播中跨帧特征的融合。在第二阶段中引入了一个辅助损失，使传播模块获得的特征保留了更多与HR空间相关的信息。在第三阶段中引入了一个细化的对齐模块，以充分利用前一阶段传播模块的特征信息。大量实验证实，PP-MSVSR在Vid4数据集性能优异，仅使用 1.45M 参数PSNR指标即可达到28.13dB。 - PP-MSVSR提供两种体积模型，开发者可根据实际场景灵活选择：PP-MSVSR（参数量1.45M）与PP-MSVSR-L（参数量7.42）。 + [PP-MSVSR](https://arxiv.org/pdf/2112.02828.pdf)提供两种体积模型，开发者可根据实际场景灵活选择：[PP-MSVSR](https://arxiv.org/pdf/2112.02828.pdf)（参数量1.45M）与[PP-MSVSR-L](https://arxiv.org/pdf/2112.02828.pdf)（参数量7.42）。 ### EDVR [EDVR](https://arxiv.org/pdf/1905.02716.pdf)模型在NTIRE19视频恢复和增强挑战赛的四个赛道中都赢得了冠军，并以巨大的优势超过了第二名。视频超分的主要难点在于（1）如何在给定大运动的情况下对齐多个帧；（2）如何有效地融合具有不同运动和模糊的不同帧。首先，为了处理大的运动，EDVR模型设计了一个金字塔级联的可变形（PCD）对齐模块，在该模块中，从粗到精的可变形卷积被使用来进行特征级的帧对齐。其次，EDVR使用了时空注意力（TSA）融合模块，该模块在时间和空间上同时应用注意力机制，以强调后续恢复的重要特征。 @@ -201,6 +201,10 @@ Vimeo90K，Vid4，UDM10测试数据集上超分性能对比 - 4. [PP-MSVSR: Multi-Stage Video Super-Resolution]() ``` - @article{ + @article{jiang2021PP-MSVSR, + author = {Jiang, Lielin and Wang, Na and Dang, Qingqing and Liu, Rui and Lai, Baohua}, + title = {PP-MSVSR: Multi-Stage Video Super-Resolution}, + booktitle = {arXiv preprint arXiv:2112.02828}, + year = {2021} } ```