DeepAI AI Chat
Log In Sign Up

DAG: Depth-Aware Guidance with Denoising Diffusion Probabilistic Models

by   Gyeongnyeon Kim, et al.
Korea University

In recent years, generative models have undergone significant advancement due to the success of diffusion models. The success of these models is often attributed to their use of guidance techniques, such as classifier and classifier-free methods, which provides effective mechanisms to trade-off between fidelity and diversity. However, these methods are not capable of guiding a generated image to be aware of its geometric configuration, e.g., depth, which hinders the application of diffusion models to areas that require a certain level of depth awareness. To address this limitation, we propose a novel guidance approach for diffusion models that uses estimated depth information derived from the rich intermediate representations of diffusion models. To do this, we first present a label-efficient depth estimation framework using the internal representations of diffusion models. At the sampling phase, we utilize two guidance techniques to self-condition the generated image using the estimated depth map, the first of which uses pseudo-labeling, and the subsequent one uses a depth-domain diffusion prior. Experiments and extensive ablation studies demonstrate the effectiveness of our method in guiding the diffusion models toward geometrically plausible image generation. Project page is available at


page 1

page 4

page 5

page 7

page 8

page 15

page 16

page 17


Classifier-Free Diffusion Guidance

Classifier guidance is a recently introduced method to trade off mode co...

Improving Sample Quality of Diffusion Models Using Self-Attention Guidance

Following generative adversarial networks (GANs), a de facto standard mo...

Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation

Denoising Diffusion Probabilistic Model (DDPM) is able to make flexible ...

NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views

Virtual reality and augmented reality (XR) bring increasing demand for 3...

Towards Practical Plug-and-Play Diffusion Models

Diffusion-based generative models have achieved remarkable success in im...

Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC

Since their introduction, diffusion models have quickly become the preva...

Monocular Depth Estimation using Diffusion Models

We formulate monocular depth estimation using denoising diffusion models...

Code Repositories