BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration

11/03/2020
by   Majed El Helou, et al.
8

Image restoration encompasses fundamental image processing tasks that have been addressed with different algorithms and deep learning methods. Classical restoration algorithms leverage a variety of priors, either implicitly or explicitly. Their priors are hand-designed and their corresponding weights are heuristically assigned. Thus, deep learning methods often produce superior restoration quality. Deep networks are, however, capable of strong and hardly-predictable hallucinations. Networks jointly and implicitly learn to be faithful to the observed data while learning an image prior, and the separation of original and hallucinated data downstream is then not possible. This limits their wide-spread adoption in restoration applications. Furthermore, it is often the hallucinated part that is victim to degradation-model overfitting. We present an approach with decoupled network-prior hallucination and data fidelity. We refer to our framework as the Bayesian Integration of a Generative Prior (BIGPrior). Our BIGPrior method is rooted in a Bayesian restoration framework, and tightly connected to classical restoration methods. In fact, our approach can be viewed as a generalization of a large family of classical restoration algorithms. We leverage a recent network inversion method to extract image prior information from a generative network. We show on image colorization, inpainting, and denoising that our framework consistently improves the prior results through good integration of data fidelity. Our method, though partly reliant on the quality of the generative network inversion, is competitive with state-of-the-art supervised and task-specific restoration methods. It also provides an additional metric that sets forth the degree of prior reliance per pixel. Indeed, the per pixel contributions of the decoupled data fidelity and prior terms are readily available in our proposed framework.

READ FULL TEXT

page 1

page 3

page 5

page 6

page 7

page 8

research
04/26/2021

Dynamic Degradation for Image Restoration and Fusion

The deep-learning-based image restoration and fusion methods have achiev...
research
03/11/2022

TAPE: Task-Agnostic Prior Embedding for Image Restoration

Learning an generalized prior for natural image restoration is an import...
research
02/28/2023

TextIR: A Simple Framework for Text-based Editable Image Restoration

Most existing image restoration methods use neural networks to learn str...
research
06/10/2023

Learning Image-Adaptive Codebooks for Class-Agnostic Image Restoration

Recent work on discrete generative priors, in the form of codebooks, has...
research
05/27/2020

Image Restoration from Parametric Transformations using Generative Models

When images are statistically described by a generative model we can use...
research
05/10/2023

Mobile Image Restoration via Prior Quantization

In digital images, the performance of optical aberration is a multivaria...
research
06/04/2022

All One Needs to Know about Priors for Deep Image Restoration and Enhancement: A Survey

Image restoration and enhancement is a process of improving the image qu...

Please sign up or login with your details

Forgot password? Click here to reset