Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding

07/21/2020
by   David Klindt, et al.
5

We construct an unsupervised learning model that achieves nonlinear disentanglement of underlying factors of variation in naturalistic videos. Previous work suggests that representations can be disentangled if all but a few factors in the environment stay constant at any point in time. As a result, algorithms proposed for this problem have only been tested on carefully constructed datasets with this exact property, leaving it unclear whether they will transfer to natural scenes. Here we provide evidence that objects in segmented natural movies undergo transitions that are typically small in magnitude with occasional large jumps, which is characteristic of a temporally sparse distribution. We leverage this finding and present SlowVAE, a model for unsupervised representation learning that uses a sparse prior on temporally adjacent observations to disentangle generative factors without any assumptions on the number of changing factors. We provide a proof of identifiability and show that the model reliably learns disentangled representations on several established benchmark datasets, often surpassing the current state-of-the-art. We additionally demonstrate transferability towards video datasets with natural dynamics, Natural Sprites and KITTI Masks, which we contribute as benchmarks for guiding disentanglement research towards more natural data domains.

READ FULL TEXT

page 8

page 22

page 24

research
07/28/2020

A Commentary on the Unsupervised Learning of Disentangled Representations

The goal of the unsupervised learning of disentangled representations is...
research
05/31/2017

Unsupervised Learning of Disentangled Representations from Video

We present a new model DrNET that learns disentangled image representati...
research
04/17/2023

Leveraging sparse and shared feature activations for disentangled representation learning

Recovering the latent factors of variation of high dimensional data has ...
research
05/29/2019

A Heuristic for Unsupervised Model Selection for Variational Disentangled Representation Learning

Disentangled representations have recently been shown to improve data ef...
research
11/02/2017

Variational Inference of Disentangled Latent Concepts from Unlabeled Observations

Disentangled representations, where the higher level data generative fac...
research
11/29/2018

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

In recent years, the interest in unsupervised learning of disentangled r...
research
08/06/2020

Learning to Factorize and Relight a City

We propose a learning-based framework for disentangling outdoor scenes i...

Please sign up or login with your details

Forgot password? Click here to reset