DeepAI AI Chat
Log In Sign Up

DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models

12/17/2022
by   Gyeongnyeon Kim, et al.
Korea University
0

In recent years, generative models have undergone significant advancement due to the success of diffusion models. The success of these models is often attributed to their use of guidance techniques, such as classifier and classifier-free methods, which provides effective mechanisms to trade-off between fidelity and diversity. However, these methods are not capable of guiding a generated image to be aware of its geometric configuration, e.g., depth, which hinders the application of diffusion models to areas that require a certain level of depth awareness. To address this limitation, we propose a novel guidance approach for diffusion models that uses estimated depth information derived from the rich intermediate representations of diffusion models. To do this, we first present a label-efficient depth estimation framework using the internal representations of diffusion models. At the sampling phase, we utilize two guidance techniques to self-condition the generated image using the estimated depth map, the first of which uses pseudo-labeling, and the subsequent one uses a depth-domain diffusion prior. Experiments and extensive ablation studies demonstrate the effectiveness of our method in guiding the diffusion models toward geometrically plausible image generation. Project page is available at https://ku-cvlab.github.io/DAG/.

READ FULL TEXT

page 1

page 4

page 5

page 7

page 8

page 15

page 16

page 17

07/26/2022

Classifier-Free Diffusion Guidance

Classifier guidance is a recently introduced method to trade off mode co...
10/03/2022

Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Following generative adversarial networks (GANs), a de facto standard mo...
06/23/2022

Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation

Denoising Diffusion Probabilistic Model (DDPM) is able to make flexible ...
11/29/2022

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views

Virtual reality and augmented reality (XR) bring increasing demand for 3...
12/12/2022

Towards Practical Plug-and-Play Diffusion Models

Diffusion-based generative models have achieved remarkable success in im...
02/22/2023

Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC

Since their introduction, diffusion models have quickly become the preva...
02/28/2023

Monocular Depth Estimation using Diffusion Models

We formulate monocular depth estimation using denoising diffusion models...

Code Repositories