A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation

10/27/2020
by   Francesco Locatello, et al.
27

The idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. In this paper, we provide a sober look at recent progress in the field and challenge some common assumptions. We first theoretically show that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data. Then, we train over 14000 models covering most prominent methods and evaluation metrics in a reproducible large-scale experimental study on eight data sets. We observe that while the different methods successfully enforce properties "encouraged" by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision. Furthermore, different evaluation metrics do not always agree on what should be considered "disentangled" and exhibit systematic differences in the estimation. Finally, increased disentanglement does not seem to necessarily lead to a decreased sample complexity of learning for downstream tasks. Our results suggest that future work on disentanglement learning should be explicit about the role of inductive biases and (implicit) supervision, investigate concrete benefits of enforcing disentanglement of the learned representations, and consider a reproducible experimental setup covering several data sets.

READ FULL TEXT

page 13

page 22

page 25

page 28

page 29

page 31

page 37

page 39

11/29/2018

Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations

In recent years, the interest in unsupervised learning of disentangled r...
07/28/2020

A Commentary on the Unsupervised Learning of Disentangled Representations

The goal of the unsupervised learning of disentangled representations is...
05/03/2019

Disentangling Factors of Variation Using Few Labels

Learning disentangled representations is considered a cornerstone proble...
06/07/2019

On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Learning meaningful and compact representations with structurally disent...
02/16/2018

Disentangling by Factorising

We define and address the problem of unsupervised learning of disentangl...
03/12/2021

VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts

Disentangled representations support a range of downstream tasks includi...
12/16/2020

Measuring Disentanglement: A Review of Metrics

Learning to disentangle and represent factors of variation in data is an...