GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement

04/20/2021
by   Martin Engelcke, et al.
7

Advances in object-centric generative models (OCGMs) have culminated in the development of a broad range of methods for unsupervised object segmentation and interpretable object-centric scene generation. These methods, however, are limited to simulated and real-world datasets with limited visual complexity. Moreover, object representations are often inferred using RNNs which do not scale well to large images or iterative refinement which avoids imposing an unnatural ordering on objects in an image but requires the a priori initialisation of a fixed number of object representations. In contrast to established paradigms, this work proposes an embedding-based approach in which embeddings of pixels are clustered in a differentiable fashion using a stochastic, non-parametric stick-breaking process. Similar to iterative refinement, this clustering procedure also leads to randomly ordered object representations, but without the need of initialising a fixed number of clusters a priori. This is used to develop a new model, GENESIS-V2, which can infer a variable number of object representations without using RNNs or iterative refinement. We show that GENESIS-V2 outperforms previous methods for unsupervised image segmentation and object-centric scene generation on established synthetic datasets as well as more complex real-world datasets.

READ FULL TEXT

page 6

page 8

page 9

page 17

page 18

page 19

page 20

page 21

research
06/07/2023

Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities

Unsupervised video-based object-centric learning is a promising avenue t...
research
05/18/2023

SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models

Object-centric learning aims to represent visual data with a set of obje...
research
07/13/2020

Reconstruction Bottlenecks in Object-Centric Generative Models

A range of methods with suitable inductive biases exist to learn interpr...
research
09/29/2022

Bridging the Gap to Real-World Object-Centric Learning

Humans naturally decompose their environment into entities at the approp...
research
05/31/2023

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

The aim of object-centric vision is to construct an explicit representat...
research
04/16/2018

IterGANs: Iterative GANs to Learn and Control 3D Object Transformation

We are interested in learning visual representations which allow for 3D ...
research
05/27/2022

Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos

Unsupervised object-centric learning aims to represent the modular, comp...

Please sign up or login with your details

Forgot password? Click here to reset