Log In Sign Up

Unsupervised Layered Image Decomposition into Object Prototypes

by   Tom Monnier, et al.

We present an unsupervised learning framework for decomposing images into layers of automatically discovered object models. Contrary to recent approaches that model image layers with autoencoder networks, we represent them as explicit transformations of a small set of prototypical images. Our model has three main components: (i) a set of object prototypes in the form of learnable images with a transparency channel, which we refer to as sprites; (ii) differentiable parametric functions predicting occlusions and transformation parameters necessary to instantiate the sprites in a given image; (iii) a layered image formation model with occlusion for compositing these instances into complete images including background. By jointly learning the sprites and occlusion/transformation predictors to reconstruct images, our approach not only yields accurate layered image decompositions, but also identifies object categories and instance parameters. We first validate our approach by providing results on par with the state of the art on standard multi-object synthetic benchmarks (Tetrominoes, Multi-dSprites, CLEVR6). We then demonstrate the applicability of our model to real images in tasks that include clustering (SVHN, GTSRB), cosegmentation (Weizmann Horse) and object discovery from unfiltered social network images. To the best of our knowledge, our approach is the first layered image decomposition algorithm that learns an explicit and shared concept of object type, and is robust enough to be applied to real images.


page 1

page 3

page 6

page 7

page 8

page 12

page 13

page 14


Unsupervised Foreground-Background Segmentation with Equivariant Layered GANs

We propose an unsupervised foreground-background segmentation method via...

Large-Scale Unsupervised Object Discovery

Existing approaches to unsupervised object discovery (UOD) do not scale ...

Occlusion-Aware Video Deblurring with a New Layered Blur Model

We present a deblurring method for scenes with occluding objects using a...

Emergence of Object Segmentation in Perturbed Generative Models

We introduce a novel framework to build a model that can learn how to se...

Unsupervised part representation by Flow Capsules

Capsule networks are designed to parse an image into a hierarchy of obje...

Unsupervised Image Decomposition with Phase-Correlation Networks

The ability to decompose scenes into their object components is a desire...

Unsupervised learning of object frames by dense equivariant image labelling

One of the key challenges of visual perception is to extract abstract mo...