Asymptotic Consistency of α-Rényi-Approximate Posteriors

02/05/2019
by   Prateek Jaiswal, et al.
0

In this work, we study consistency properties of α-Rényi approximate posteriors, a class of variational Bayesian methods that approximate an intractable Bayesian posterior with a member of a tractable family of distributions, the latter chosen to minimize the α-Rényi divergence from the true posterior. Unique to our work is that we consider settings with α > 1, resulting in approximations that upperbound the log-likelihood, and result in approximations with a wider spread than traditional variational approaches that minimize the Kullback-Liebler divergence from the posterior. We provide sufficient conditions under which consistency holds, centering around the existence of a 'good' sequence of distributions in the approximating family. We discuss examples where this holds and show how the existence of such a good sequence implies posterior consistency in the limit of an infinite number of observations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/28/2012

Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression

We propose a general algorithm for approximating nonstandard Bayesian po...
research
05/13/2019

Variational approximations using Fisher divergence

Modern applications of Bayesian inference involve models that are suffic...
research
02/07/2023

Structured variational approximations with skew normal decomposable graphical models

Although there is much recent work developing flexible variational metho...
research
11/04/2019

Asymptotic Consistency of Loss-Calibrated Variational Bayes

This paper establishes the asymptotic consistency of the loss-calibrate...
research
11/20/2016

Variational Boosting: Iteratively Refining Posterior Approximations

We propose a black-box variational inference method to approximate intra...
research
04/27/2015

On Sparse variational methods and the Kullback-Leibler divergence between stochastic processes

The variational framework for learning inducing variables (Titsias, 2009...
research
12/01/2014

Sparse Variational Bayesian Approximations for Nonlinear Inverse Problems: applications in nonlinear elastography

This paper presents an efficient Bayesian framework for solving nonlinea...

Please sign up or login with your details

Forgot password? Click here to reset