SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models

05/18/2023
by   Ziyi Wu, et al.
0

Object-centric learning aims to represent visual data with a set of object entities (a.k.a. slots), providing structured representations that enable systematic generalization. Leveraging advanced architectures like Transformers, recent approaches have made significant progress in unsupervised object discovery. In addition, slot-based representations hold great potential for generative modeling, such as controllable image generation and object manipulation in image editing. However, current slot-based methods often produce blurry images and distorted objects, exhibiting poor generative modeling capabilities. In this paper, we focus on improving slot-to-image decoding, a crucial aspect for high-quality visual generation. We introduce SlotDiffusion – an object-centric Latent Diffusion Model (LDM) designed for both image and video data. Thanks to the powerful modeling capacity of LDMs, SlotDiffusion surpasses previous slot models in unsupervised object segmentation and visual generation across six datasets. Furthermore, our learned object features can be utilized by existing object-centric dynamics models, improving video prediction quality and downstream temporal reasoning tasks. Finally, we demonstrate the scalability of SlotDiffusion to unconstrained real-world datasets such as PASCAL VOC and COCO, when integrated with self-supervised pre-trained image encoders.

READ FULL TEXT

page 6

page 7

page 8

page 9

page 16

page 21

page 22

page 23

research
06/07/2023

Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities

Unsupervised video-based object-centric learning is a promising avenue t...
research
07/18/2020

Slot Contrastive Networks: A Contrastive Approach for Representing Objects

Unsupervised extraction of objects from low-level visual data is an impo...
research
09/29/2022

Bridging the Gap to Real-World Object-Centric Learning

Humans naturally decompose their environment into entities at the approp...
research
06/01/2023

Rotating Features for Object Discovery

The binding problem in human cognition, concerning how the brain represe...
research
04/20/2021

GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement

Advances in object-centric generative models (OCGMs) have culminated in ...
research
10/05/2020

Improving Generative Imagination in Object-Centric World Models

The remarkable recent advances in object-centric generative world models...
research
03/20/2023

Object-Centric Slot Diffusion

Despite remarkable recent advances, making object-centric learning work ...

Please sign up or login with your details

Forgot password? Click here to reset