ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models

by   Jooyoung Choi, et al.

Denoising diffusion probabilistic models (DDPM) have shown remarkable performance in unconditional image generation. However, due to the stochasticity of the generative process in DDPM, it is challenging to generate images with the desired semantics. In this work, we propose Iterative Latent Variable Refinement (ILVR), a method to guide the generative process in DDPM to generate high-quality images based on a given reference image. Here, the refinement of the generative process in DDPM enables a single DDPM to sample images from various sets directed by the reference image. The proposed ILVR method generates high-quality images while controlling the generation. The controllability of our method allows adaptation of a single DDPM without any additional learning in various image generation tasks, such as generation from various downsampling factors, multi-domain image translation, paint-to-image, and editing with scribbles.


page 1

page 5

page 6

page 7

page 9

page 12

page 13

page 14


Nested Diffusion Processes for Anytime Image Generation

Diffusion models are the current state-of-the-art in image generation, s...

Text2Layer: Layered Image Generation using Latent Diffusion Model

Layer compositing is one of the most popular image editing workflows amo...

Guided Image Synthesis via Initial Image Editing in Diffusion Model

Diffusion models have the ability to generate high quality images by den...

Cartoondiff: Training-free Cartoon Image Generation with Diffusion Transformer Models

Image cartoonization has attracted significant interest in the field of ...

Generative Probabilistic Image Colorization

We propose Generative Probabilistic Image Colorization, a diffusion-base...

Latent Painter

Latent diffusers revolutionized the generative AI and inspired creative ...

f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation

Diffusion models (DMs) have recently emerged as SoTA tools for generativ...

Please sign up or login with your details

Forgot password? Click here to reset