Diffusion Autoencoders: Toward a Meaningful and Decodable Representation

11/30/2021
by   Konpat Preechakul, et al.
0

Diffusion probabilistic models (DPMs) have achieved remarkable quality in image generation that rivals GANs'. But unlike GANs, DPMs use a set of latent variables that lack semantic meaning and cannot serve as a useful representation for other tasks. This paper explores the possibility of using DPMs for representation learning and seeks to extract a meaningful and decodable representation of an input image via autoencoding. Our key idea is to use a learnable encoder for discovering the high-level semantics, and a DPM as the decoder for modeling the remaining stochastic variations. Our method can encode any image into a two-part latent code, where the first part is semantically meaningful and linear, and the second part captures stochastic details, allowing near-exact reconstruction. This capability enables challenging applications that currently foil GAN-based methods, such as attribute manipulation on real images. We also show that this two-level encoding improves denoising efficiency and naturally facilitates various downstream tasks including few-shot conditional sampling. Please visit our project page: https://Diff-AE.github.io/

READ FULL TEXT

page 5

page 6

page 13

page 16

page 17

page 18

page 19

page 20

research
10/20/2022

Representation Learning with Diffusion Models

Diffusion models (DMs) have achieved state-of-the-art results for image ...
research
03/07/2021

Repurposing GANs for One-shot Semantic Part Segmentation

While GANs have shown success in realistic image generation, the idea of...
research
07/12/2023

DiffuseGAE: Controllable and High-fidelity Image Manipulation from Disentangled Representation

Diffusion probabilistic models (DPMs) have shown remarkable results on v...
research
05/21/2022

Improvements to Self-Supervised Representation Learning for Masked Image Modeling

This paper explores improvements to the masked image modeling (MIM) para...
research
12/26/2022

Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models

Diffusion Probabilistic Models (DPMs) have shown a powerful capacity of ...
research
11/07/2017

Theoretical limitations of Encoder-Decoder GAN architectures

Encoder-decoder GANs architectures (e.g., BiGAN and ALI) seek to add an ...
research
10/10/2022

f-DM: A Multi-stage Diffusion Model via Progressive Signal Transformation

Diffusion models (DMs) have recently emerged as SoTA tools for generativ...

Please sign up or login with your details

Forgot password? Click here to reset