Mixture of Discrete Normalizing Flows for Variational Inference

06/28/2020
by   Tomasz Kuśmierczyk, et al.
0

Advances in gradient-based inference have made distributional approximations for posterior distribution of latent-variable models easy, but only for continuous latent spaces. Models with discrete latent variables still require analytic marginalization, continuous relaxations, or specialized algorithms that are difficult to generalize already for minor variations of the model. Discrete normalizing flows could, in principle, be used as approximations while allowing efficient gradient-based learning, but as explained in this work they are not sufficiently expressive for representing realistic posterior distributions even for simple cases. We overcome this limitation by considering mixtures of discrete normalizing flows instead, and present a novel algorithm for modeling the posterior distribution of models with discrete latent variables, based on boosting variational inference.

READ FULL TEXT
research
11/07/2015

Hierarchical Variational Models

Black box variational inference allows researchers to easily prototype a...
research
06/11/2020

Latent Transformations for Discrete-Data Normalising Flows

Normalising flows (NFs) for discrete data are challenging because parame...
research
02/15/2020

Latent Variable Modelling with Hyperbolic Normalizing Flows

The choice of approximate posterior distributions plays a central role i...
research
05/28/2018

Discrete flow posteriors for variational inference in discrete dynamical systems

Each training step for a variational autoencoder (VAE) requires us to sa...
research
03/18/2019

A RAD approach to deep mixture models

Flow based models such as Real NVP are an extremely powerful approach to...
research
12/01/2016

Piecewise Latent Variables for Neural Variational Text Processing

Advances in neural variational inference have facilitated the learning o...
research
06/24/2022

A Phylogenetic Model of the Evolution of Discrete Matrices for the Joint Inference of Lexical and Phonological Language Histories

We propose a model of the evolution of a matrix along a phylogenetic tre...

Please sign up or login with your details

Forgot password? Click here to reset