Adaptive variational Bayes: Optimality, computation and applications

09/07/2021
by   Ilsang Ohn, et al.
15

In this paper, we explore adaptive inference based on variational Bayes. Although a number of studies have been conducted to analyze contraction properties of variational posteriors, there is still a lack of a general and computationally tractable variational Bayes method that can achieve adaptive optimal contraction of the variational posterior. We propose a novel variational Bayes framework, called adaptive variational Bayes, which can operate on a collection of models with varying dimensions and structures. The proposed framework combines variational posteriors over individual models with certain weights to obtain a variational posterior over the entire model. It turns out that this combined variational posterior minimizes the Kullback-Leibler divergence to the original posterior distribution. We show that the proposed variational posterior achieves optimal contraction rates adaptively under very general conditions and attains model selection consistency when the true model structure exists. We apply the general results obtained for the adaptive variational Bayes to several examples including deep learning models and derive some new and adaptive inference results. Moreover, we consider the use of quasi-likelihood in our framework. We formulate conditions on the quasi-likelihood to ensure the adaptive optimality and discuss specific applications to stochastic block models and nonparametric regression with sub-Gaussian errors.

READ FULL TEXT
research
09/03/2021

Variational Bayes algorithm and posterior consistency of Ising model parameter estimation

Ising models originated in statistical physics and are widely used in mo...
research
01/13/2014

A variational Bayes framework for sparse adaptive estimation

Recently, a number of mostly ℓ_1-norm regularized least squares type det...
research
09/22/2021

Contraction rates for sparse variational approximations in Gaussian process regression

We study the theoretical properties of a variational Bayes method in the...
research
12/23/2022

Statistical inference with normal-compound gamma priors in regression models

Scale-mixture shrinkage priors have recently been shown to possess robus...
research
03/31/2019

Robust inference for general framework of projection structures

We develop a general framework of projection structures and study the pr...
research
03/21/2023

Bayesian nonparametric spectral analysis of locally stationary processes

Based on a novel dynamic Whittle likelihood approximation for locally st...
research
01/06/2020

Variational Bayesian Methods for Stochastically Constrained System Design Problems

We study system design problems stated as parameterized stochastic progr...

Please sign up or login with your details

Forgot password? Click here to reset