DeepAI AI Chat
Log In Sign Up

Fast Dual Variational Inference for Non-Conjugate LGMs

by   Mohammad Emtiyaz Khan, et al.

Latent Gaussian models (LGMs) are widely used in statistics and machine learning. Bayesian inference in non-conjugate LGMs is difficult due to intractable integrals involving the Gaussian prior and non-conjugate likelihoods. Algorithms based on variational Gaussian (VG) approximations are widely employed since they strike a favorable balance between accuracy, generality, speed, and ease of use. However, the structure of the optimization problems associated with these approximations remains poorly understood, and standard solvers take too long to converge. We derive a novel dual variational inference approach that exploits the convexity property of the VG approximations. We obtain an algorithm that solves a convex optimization problem, reduces the number of variational parameters, and converges much faster than previous methods. Using real-world data, we demonstrate these advantages on a variety of LGMs, including Gaussian process classification, and latent Gaussian Markov random fields.


page 1

page 2

page 3

page 4


Variational Inference for Gaussian Process Modulated Poisson Processes

We present the first fully variational Bayesian inference scheme for con...

A Deterministic Global Optimization Method for Variational Inference

Variational inference methods for latent variable statistical models hav...

Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations

Natural-gradient methods enable fast and simple algorithms for variation...

Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models

Bayesian inference plays an important role in advancing machine learning...

Fitting latent non-Gaussian models using variational Bayes and Laplace approximations

Latent Gaussian models (LGMs) are perhaps the most commonly used class o...

Latent Gaussian Model Boosting

Latent Gaussian models and boosting are widely used techniques in statis...

Approximate Inference with the Variational Holder Bound

We introduce the Variational Holder (VH) bound as an alternative to Vari...