Tensorizing flows: a tool for variational inference

05/03/2023
by   Yuehaw Khoo, et al.
0

Fueled by the expressive power of deep neural networks, normalizing flows have achieved spectacular success in generative modeling, or learning to draw new samples from a distribution given a finite dataset of training samples. Normalizing flows have also been applied successfully to variational inference, wherein one attempts to learn a sampler based on an expression for the log-likelihood or energy function of the distribution, rather than on data. In variational inference, the unimodality of the reference Gaussian distribution used within the normalizing flow can cause difficulties in learning multimodal distributions. We introduce an extension of normalizing flows in which the Gaussian reference is replaced with a reference distribution that is constructed via a tensor network, specifically a matrix product state or tensor train. We show that by combining flows with tensor networks on difficult variational inference tasks, we can improve on the results obtained by using either tool without the other.

READ FULL TEXT

page 16

page 20

page 21

research
03/15/2018

Sylvester Normalizing Flows for Variational Inference

Variational inference relies on flexible approximate posterior distribut...
research
07/10/2020

Variational Inference with Continuously-Indexed Normalizing Flows

Continuously-indexed flows (CIFs) have recently achieved improvements ov...
research
02/22/2020

VFlow: More Expressive Generative Flows with Variational Data Augmentation

Generative flows are promising tractable models for density modeling tha...
research
09/08/2020

Learning more expressive joint distributions in multimodal variational methods

Data often are formed of multiple modalities, which jointly describe the...
research
04/04/2022

Discretely Indexed Flows

In this paper we propose Discretely Indexed flows (DIF) as a new tool fo...
research
02/11/2020

Large Scale Tensor Regression using Kernels and Variational Inference

We outline an inherent weakness of tensor factorization models when late...
research
07/05/2018

Learning in Variational Autoencoders with Kullback-Leibler and Renyi Integral Bounds

In this paper we propose two novel bounds for the log-likelihood based o...

Please sign up or login with your details

Forgot password? Click here to reset