Representation Disentaglement via Regularization by Identification

02/28/2023
by   Juan Castorena, et al.
0

This work focuses on the problem of learning disentangled representations from observational data. Given observations 𝐱^(i) for i=1,...,N drawn from p(𝐱|𝐲) with generative variables 𝐲 admitting the distribution factorization p(𝐲) = ∏_c p(𝐲_c ) we ask whether learning disentangled representations matching the space of observations with identification guarantees on the posterior p(𝐳| 𝐱, 𝐲̂_c) for each c, is plausible. We argue modern deep representation learning models are ill-posed with collider bias behaviour; a source of bias producing entanglement between generating variables. Under the rubric of causality, we show this issue can be explained and reconciled under the condition of identifiability; attainable under supervision or a weak-form of it. For this, we propose regularization by identification (ReI), a regularization framework defined by the identification of the causal queries involved in the learning problem. Empirical evidence shows that enforcing ReI in a variational framework results in disentangled representations equipped with generalization capabilities to out-of-distribution examples and that aligns nicely with the true expected effect between generating variables and measurement apparatus.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/23/2021

Group-disentangled Representation Learning with Weakly-Supervised Regularization

Learning interpretable and human-controllable representations that uncov...
research
12/05/2022

Identification of Unobservables in Observations

In empirical studies, the data usually don't include all the variables o...
research
09/08/2021

Desiderata for Representation Learning: A Causal Perspective

Representation learning constructs low-dimensional representations to su...
research
08/22/2020

WeLa-VAE: Learning Alternative Disentangled Representations Using Weak Labels

Learning disentangled representations without supervision or inductive b...
research
10/19/2020

Learning Optimal Conditional Priors For Disentangled Representations

A large part of the literature on learning disentangled representations ...
research
10/22/2019

Weakly Supervised Disentanglement with Guarantees

Learning disentangled representations that correspond to factors of vari...
research
08/31/2021

Disentanglement Analysis with Partial Information Decomposition

Given data generated from multiple factors of variation that cooperative...

Please sign up or login with your details

Forgot password? Click here to reset