Log In Sign Up

Training a Better Loss Function for Image Restoration

by   Aamir Mustafa, et al.

Central to the application of neural networks in image restoration problems, such as single image super resolution, is the choice of a loss function that encourages natural and perceptually pleasing results. A popular choice for a loss function is a pre-trained network, such as VGG and LPIPS, which is used as a feature extractor for computing the difference between restored and reference images. However, such an approach has multiple drawbacks: it is computationally expensive, requires regularization and hyper-parameter tuning, and involves a large network trained on an unrelated task. In this work, we explore the question of what makes a good loss function for an image restoration task. First, we observe that a single natural image is sufficient to train a lightweight feature extractor that outperforms state-of-the-art loss functions in single image super resolution, denoising, and JPEG artefact removal. We propose a novel Multi-Scale Discriminative Feature (MDF) loss comprising a series of discriminators, trained to penalize errors introduced by a generator. Second, we show that an effective loss function does not have to be a good predictor of perceived image quality, but instead needs to be specialized in identifying the distortions for a given restoration method.


page 4

page 6

page 8

page 9

page 10

page 11

page 12


Single Image Super-Resolution with Dilated Convolution based Multi-Scale Information Learning Inception Module

Traditional works have shown that patches in a natural image tend to red...

Generative Collaborative Networks for Single Image Super-Resolution

A common issue of deep neural networks-based methods for the problem of ...

Image quality measurements and denoising using Fourier Ring Correlations

Image quality is a nebulous concept with different meanings to different...

Feature Forwarding for Efficient Single Image Dehazing

Haze degrades content and obscures information of images, which can nega...

Learning the Loss Functions in a Discriminative Space for Video Restoration

With more advanced deep network architectures and learning schemes such ...

Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model

Most existing Image Restoration (IR) models are task-specific, which can...