Disentangled Representations in Neural Models

02/07/2016
by   William Whitney, et al.
0

Representation learning is the foundation for the recent success of neural network models. However, the distributed representations generated by neural networks are far from ideal. Due to their highly entangled nature, they are di cult to reuse and interpret, and they do a poor job of capturing the sparsity which is present in real- world transformations. In this paper, I describe methods for learning disentangled representations in the two domains of graphics and computation. These methods allow neural methods to learn representations which are easy to interpret and reuse, yet they incur little or no penalty to performance. In the Graphics section, I demonstrate the ability of these methods to infer the generating parameters of images and rerender those images under novel conditions. In the Computation section, I describe a model which is able to factorize a multitask learning problem into subtasks and which experiences no catastrophic forgetting. Together these techniques provide the tools to design a wide range of models that learn disentangled representations and better model the factors of variation in the real world.

READ FULL TEXT

page 28

page 29

page 30

page 33

research
11/26/2019

Independence Promoted Graph Disentangled Networks

We address the problem of disentangled representation learning with inde...
research
09/21/2020

Improving Robustness and Generality of NLP Models Using Disentangled Representations

Supervised neural networks, which first map an input x to a single repre...
research
10/12/2016

Deep disentangled representations for volumetric reconstruction

We introduce a convolutional neural network for inferring a compact dise...
research
05/29/2019

Are Disentangled Representations Helpful for Abstract Visual Reasoning?

A disentangled representation encodes information about the salient fact...
research
06/07/2019

On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset

Learning meaningful and compact representations with structurally disent...
research
08/26/2021

A Tutorial on Learning Disentangled Representations in the Imaging Domain

Disentangled representation learning has been proposed as an approach to...
research
12/06/2019

Achieving Robustness in the Wild via Adversarial Mixing with Disentangled Representations

Recent research has made the surprising finding that state-of-the-art de...

Please sign up or login with your details

Forgot password? Click here to reset