DeepAI AI Chat
Log In Sign Up

Recent Advances in Autoencoder-Based Representation Learning

by   Michael Tschannen, et al.
ETH Zurich

Learning useful representations with little or no supervision is a key challenge in artificial intelligence. We provide an in-depth review of recent advances in representation learning with a focus on autoencoder-based models. To organize these results we make use of meta-priors believed useful for downstream tasks, such as disentanglement and hierarchical organization of features. In particular, we uncover three main mechanisms to enforce such properties, namely (i) regularizing the (approximate or aggregate) posterior distribution, (ii) factorizing the encoding and decoding distribution, or (iii) introducing a structured prior distribution. While there are some promising results, implicit or explicit supervision remains a key enabler and all current methods use strong inductive biases and modeling assumptions. Finally, we provide an analysis of autoencoder-based representation learning through the lens of rate-distortion theory and identify a clear tradeoff between the amount of prior knowledge available about the downstream tasks, and how useful the representation is for this task.


page 3

page 8


Representation Learning for Out-Of-Distribution Generalization in Reinforcement Learning

Learning data representations that are useful for various downstream tas...

Boosting Object Representation Learning via Motion and Object Continuity

Recent unsupervised multi-object detection models have shown impressive ...

Gromov-Wasserstein Autoencoders

Learning concise data representations without supervisory signals is a f...

dpVAEs: Fixing Sample Generation for Regularized VAEs

Unsupervised representation learning via generative modeling is a staple...

A Survey on Self-Supervised Representation Learning

Learning meaningful representations is at the heart of many tasks in the...

Neural PCA for Flow-Based Representation Learning

Of particular interest is to discover useful representations solely from...

Neocortical plasticity: an unsupervised cake but no free lunch

The fields of artificial intelligence and neuroscience have a long histo...