Fourier Space Losses for Efficient Perceptual Image Super-Resolution

06/01/2021
by   Dario Fuoli, et al.
0

Many super-resolution (SR) models are optimized for high performance only and therefore lack efficiency due to large model complexity. As large models are often not practical in real-world applications, we investigate and propose novel loss functions, to enable SR with high perceptual quality from much more efficient models. The representative power for a given low-complexity generator network can only be fully leveraged by strong guidance towards the optimal set of parameters. We show that it is possible to improve the performance of a recently introduced efficient generator architecture solely with the application of our proposed loss functions. In particular, we use a Fourier space supervision loss for improved restoration of missing high-frequency (HF) content from the ground truth image and design a discriminator architecture working directly in the Fourier domain to better match the target HF distribution. We show that our losses' direct emphasis on the frequencies in Fourier-space significantly boosts the perceptual image quality, while at the same time retaining high restoration quality in comparison to previously proposed loss functions for this task. The performance is further improved by utilizing a combination of spatial and frequency domain losses, as both representations provide complementary information during training. On top of that, the trained generator achieves comparable results with and is 2.4x and 48x faster than state-of-the-art perceptual SR methods RankSRGAN and SRFlow respectively.

READ FULL TEXT

page 4

page 8

page 12

page 13

page 14

research
07/23/2020

Frequency Domain-based Perceptual Loss for Super Resolution

We introduce Frequency Domain Perceptual Loss (FDPL), a loss function fo...
research
07/22/2023

On the Effectiveness of Spectral Discriminators for Perceptual Quality Improvement

Several recent studies advocate the use of spectral discriminators, whic...
research
03/20/2020

Learning the Loss Functions in a Discriminative Space for Video Restoration

With more advanced deep network architectures and learning schemes such ...
research
06/22/2023

Restoration of the JPEG Maximum Lossy Compressed Face Images with Hourglass Block based on Early Stopping Discriminator

When a JPEG image is compressed using the loss compression method with a...
research
03/01/2023

Neural inverse procedural modeling of knitting yarns from images

We investigate the capabilities of neural inverse procedural modeling to...
research
06/19/2021

One-to-many Approach for Improving Super-Resolution

Super-resolution (SR) is a one-to-many task with multiple possible solut...
research
11/19/2018

Learning to synthesize: splitting and recombining low and high spatial frequencies for image recovery

Deep Neural Network (DNN)-based image reconstruction, despite many succe...

Please sign up or login with your details

Forgot password? Click here to reset