Statistical Inference in Mean-Field Variational Bayes

by   Wei Han, et al.

We conduct non-asymptotic analysis on the mean-field variational inference for approximating posterior distributions in complex Bayesian models that may involve latent variables. We show that the mean-field approximation to the posterior can be well-approximated relative to the Kullback-Leibler divergence discrepancy measure by a normal distribution whose center is the maximum likelihood estimator (MLE). In particular, our results imply that the center of the mean-field approximation matches the MLE up to higher-order terms and there is essentially no loss of efficiency in using it as a point estimator for the parameter in any regular parametric model with latent variables. We also propose a new class of variational weighted likelihood bootstrap (VWLB) methods for quantifying the uncertainty in the mean-field variational inference. The proposed VWLB can be viewed as a new sampling scheme that produces independent samples for approximating the posterior. Comparing with traditional sampling algorithms such Markov Chain Monte Carlo, VWLB can be implemented in parallel and is free of tuning.


Fitting Structural Equation Models via Variational Approximations

Structural equation models are commonly used to capture the structural r...

Forward Amortized Inference for Likelihood-Free Variational Marginalization

In this paper, we introduce a new form of amortized variational inferenc...

Multi-Modal Mean-Fields via Cardinality-Based Clamping

Mean Field inference is central to statistical physics. It has attracted...

α-Variational Inference with Statistical Guarantees

We propose a variational approximation to Bayesian posterior distributio...

Variational approximations using Fisher divergence

Modern applications of Bayesian inference involve models that are suffic...

Try Depth Instead of Weight Correlations: Mean-field is a Less Restrictive Assumption for Deeper Networks

We challenge the longstanding assumption that the mean-field approximati...

An Instability in Variational Inference for Topic Models

Topic models are Bayesian models that are frequently used to capture the...

Please sign up or login with your details

Forgot password? Click here to reset