DeepAI AI Chat
Log In Sign Up

ADIR: Adaptive Diffusion for Image Reconstruction

by   Shady Abu Hussein, et al.
Tel Aviv University

In recent years, denoising diffusion models have demonstrated outstanding image generation performance. The information on natural images captured by these models is useful for many image reconstruction applications, where the task is to restore a clean image from its degraded observations. In this work, we propose a conditional sampling scheme that exploits the prior learned by diffusion models while retaining agreement with the observations. We then combine it with a novel approach for adapting pretrained diffusion denoising networks to their input. We examine two adaption strategies: the first uses only the degraded image, while the second, which we advocate, is performed using images that are “nearest neighbors” of the degraded image, retrieved from a diverse dataset using an off-the-shelf visual-language model. To evaluate our method, we test it on two state-of-the-art publicly available diffusion models, Stable Diffusion and Guided Diffusion. We show that our proposed `adaptive diffusion for image reconstruction' (ADIR) approach achieves a significant improvement in the super-resolution, deblurring, and text-based editing tasks.


page 1

page 4

page 6

page 7

page 12

page 13

page 14


RenderDiffusion: Text Generation as Image Generation

Diffusion models have become a new generative paradigm for text generati...

Bi-Noising Diffusion: Towards Conditional Diffusion Models with Generative Restoration Priors

Conditional diffusion probabilistic models can model the distribution of...

A Conditional Denoising Diffusion Probabilistic Model for Radio Interferometric Image Reconstruction

In radio astronomy, signals from radio telescopes are transformed into i...

Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models

Diffusion models have become a popular approach for image generation and...

SUD^2: Supervision by Denoising Diffusion Models for Image Reconstruction

Many imaging inverse problemsx2014such as image-dependent in-painting an...

Reliability-based Mesh-to-Grid Image Reconstruction

This paper presents a novel method for the reconstruction of images from...

Ultrasound Image Reconstruction with Denoising Diffusion Restoration Models

Ultrasound image reconstruction can be approximately cast as a linear in...