Probing the Robustness of Independent Mechanism Analysis for Representation Learning

07/13/2022
by   Joanna Sliwa, et al.
21

One aim of representation learning is to recover the original latent code that generated the data, a task which requires additional information or inductive biases. A recently proposed approach termed Independent Mechanism Analysis (IMA) postulates that each latent source should influence the observed mixtures independently, complementing standard nonlinear independent component analysis, and taking inspiration from the principle of independent causal mechanisms. While it was shown in theory and experiments that IMA helps recovering the true latents, the method's performance was so far only characterized when the modeling assumptions are exactly satisfied. Here, we test the method's robustness to violations of the underlying assumptions. We find that the benefits of IMA-based regularization for recovering the true sources extend to mixing functions with various degrees of violation of the IMA principle, while standard regularizers do not provide the same merits. Moreover, we show that unregularized maximum likelihood recovers mixing functions which systematically deviate from the IMA principle, and provide an argument elucidating the benefits of IMA-based regularization.

READ FULL TEXT

page 4

page 8

research
06/09/2021

Independent mechanism analysis, a new concept?

Independent component analysis provides a principled framework for unsup...
research
06/06/2022

Embrace the Gap: VAEs Perform Independent Mechanism Analysis

Variational autoencoders (VAEs) are a popular framework for modeling com...
research
05/26/2023

Causal Component Analysis

Independent Component Analysis (ICA) aims to recover independent latent ...
research
07/21/2021

Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA

It can be argued that finding an interpretable low-dimensional represent...
research
05/16/2019

The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA

We consider the problem of recovering a common latent source with indepe...
research
10/29/2021

Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning

A key goal of unsupervised representation learning is "inverting" a data...
research
01/24/2019

Overcomplete Independent Component Analysis via SDP

We present a novel algorithm for overcomplete independent components ana...

Please sign up or login with your details

Forgot password? Click here to reset