α-Variational Inference with Statistical Guarantees

10/09/2017
by   Yun Yang, et al.
0

We propose a variational approximation to Bayesian posterior distributions, called α-VB, with provable statistical guarantees for models with and without latent variables. The standard variational approximation is a special case of α-VB with α=1. When α∈(0,1), a novel class of variational inequalities are developed for linking the Bayes risk under the variational approximation to the objective function in the variational optimization problem, implying that maximizing the evidence lower bound in variational inference has the effect of minimizing the Bayes risk within the variational density family. Operating in a frequentist setup, the variational inequalities imply that point estimates constructed from the α-VB procedure converge at an optimal rate to the true parameter in a wide range of problems. We illustrate our general theory with a number of examples, including the mean-field variational approximation to (low)-high-dimensional Bayesian linear regression with spike and slab priors, mixture of Gaussian models, latent Dirichlet allocation, and (mixture of) Gaussian variational approximation in regular parametric models.

READ FULL TEXT

page 27

page 28

research
12/25/2017

On Statistical Optimality of Variational Bayes

The article addresses a long-standing open problem on the justification ...
research
11/04/2019

Statistical Inference in Mean-Field Variational Bayes

We conduct non-asymptotic analysis on the mean-field variational inferen...
research
04/15/2019

Variational Bayes for high-dimensional linear regression with sparse priors

We study a mean-field variational Bayes (VB) approximation to Bayesian m...
research
10/23/2020

From the Expectation Maximisation Algorithm to Autoencoded Variational Bayes

Although the expectation maximisation (EM) algorithm was introduced in 1...
research
02/13/2019

On the Convergence of Extended Variational Inference for Non-Gaussian Statistical Models

Variational inference (VI) is a widely used framework in Bayesian estima...
research
07/25/2013

Streaming Variational Bayes

We present SDA-Bayes, a framework for (S)treaming, (D)istributed, (A)syn...
research
05/18/2018

Bayesian model reduction

This paper reviews recent developments in statistical structure learning...

Please sign up or login with your details

Forgot password? Click here to reset