Scalable Object-Oriented Sequential Generative Models

by   Jindong Jiang, et al.

The main limitation of previous approaches to unsupervised sequential object-oriented representation learning is in scalability. Most of the previous models have been shown to work only on scenes with a few objects. In this paper, we propose SCALOR, a generative model for SCALable sequential Object-oriented Representation. With the proposed spatially-parallel attention and proposal-rejection mechanism, SCALOR can deal with orders of magnitude more number of objects compared to the current state-of-the-art models. Besides, we introduce the background model so that SCALOR can model complex background along with many foreground objects. We demonstrate that SCALOR can deal with crowded scenes containing nearly a hundred objects while modeling complex background as well. Importantly, SCALOR is the first unsupervised model demonstrating its working in natural scenes containing several tens of moving objects.


page 7

page 8

page 9

page 14

page 15

page 16

page 17

page 18


SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decomposition

The ability to decompose complex multi-object scenes into meaningful abs...

Unsupervised Object Learning via Common Fate

Learning generative object models from unlabelled videos is a long stand...

SPOOK: A System for Probabilistic Object-Oriented Knowledge Representation

In previous work, we pointed out the limitations of standard Bayesian ne...

Guided Attention Network for Object Detection and Counting on Drones

Object detection and counting are related but challenging problems, espe...

Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking

The ability to detect and track objects in the visual world is a crucial...

A Learned Representation for Scalable Vector Graphics

Dramatic advances in generative models have resulted in near photographi...

Background Invariance Testing According to Semantic Proximity

In many applications, machine learned (ML) models are required to hold s...

Code Repositories


Official PyTorch implementation of "SCALOR: Generative World Models with Scalable Object Representations"

view repo