Learning Structured Latent Factors from Dependent Data:A Generative Model Framework from Information-Theoretic Perspective

07/21/2020
by   Ruixiang Zhang, et al.
0

Learning controllable and generalizable representation of multivariate data with desired structural properties remains a fundamental problem in machine learning. In this paper, we present a novel framework for learning generative models with various underlying structures in the latent space. We represent the inductive bias in the form of mask variables to model the dependency structure in the graphical model and extend the theory of multivariate information bottleneck to enforce it. Our model provides a principled approach to learn a set of semantically meaningful latent factors that reflect various types of desired structures like capturing correlation or encoding invariance, while also offering the flexibility to automatically estimate the dependency structure from data. We show that our framework unifies many existing generative models and can be applied to a variety of tasks including multi-modal data modeling, algorithmic fairness, and invariant risk minimization.

READ FULL TEXT

page 7

page 18

research
04/10/2018

Graphical Generative Adversarial Networks

We propose Graphical Generative Adversarial Networks (Graphical-GAN) to ...
research
06/25/2021

InteL-VAEs: Adding Inductive Biases to Variational Auto-Encoders via Intermediary Latents

We introduce a simple and effective method for learning VAEs with contro...
research
02/06/2019

Latent Space Cartography: Generalised Metric-Inspired Measures and Measure-Based Transformations for Generative Models

Deep generative models are universal tools for learning data distributio...
research
10/01/2022

Multi-objective Deep Data Generation with Correlated Property Control

Developing deep generative models has been an emerging field due to the ...
research
11/04/2020

Latent Causal Invariant Model

Current supervised learning can learn spurious correlation during the da...
research
11/25/2021

Learning Conditional Invariance through Cycle Consistency

Identifying meaningful and independent factors of variation in a dataset...
research
04/25/2023

Latent Traversals in Generative Models as Potential Flows

Despite the significant recent progress in deep generative models, the u...

Please sign up or login with your details

Forgot password? Click here to reset