The FMRIB Variational Bayesian Inference Tutorial II: Stochastic Variational Bayes

Bayesian methods have proved powerful in many applications for the inference of model parameters from data. These methods are based on Bayes' theorem, which itself is deceptively simple. However, in practice the computations required are intractable even for simple cases. Hence methods for Bayesian inference have historically either been significantly approximate, e.g., the Laplace approximation, or achieve samples from the exact solution at significant computational expense, e.g., Markov Chain Monte Carlo methods. Since around the year 2000 so-called Variational approaches to Bayesian inference have been increasingly deployed. In its most general form Variational Bayes (VB) involves approximating the true posterior probability distribution via another more 'manageable' distribution, the aim being to achieve as good an approximation as possible. In the original FMRIB Variational Bayes tutorial we documented an approach to VB based that took a 'mean field' approach to forming the approximate posterior, required the conjugacy of prior and likelihood, and exploited the Calculus of Variations, to derive an iterative series of update equations, akin to Expectation Maximisation. In this tutorial we revisit VB, but now take a stochastic approach to the problem that potentially circumvents some of the limitations imposed by the earlier methodology. This new approach bears a lot of similarity to, and has benefited from, computational methods applied to machine learning algorithms. Although, what we document here is still recognisably Bayesian inference in the classic sense, and not an attempt to use machine learning as a black-box to solve the inference problem.

Authors

• 2 publications
• 2 publications
• Variational inference for large-scale models of discrete choice

Discrete choice models are commonly used by applied statisticians in num...
12/15/2007 ∙ by Michael Braun, et al. ∙ 0

• Stochastic Variational Bayesian Inference for a Nonlinear Forward Model

Variational Bayes (VB) has been used to facilitate the calculation of th...
07/03/2020 ∙ by Michael A. Chappell, et al. ∙ 0

• Variational Bayes Inference in Digital Receivers

The digital telecommunications receiver is an important context for infe...
11/03/2018 ∙ by Viet Hung Tran, et al. ∙ 0

• A Variational Bayes Approach to Decoding in a Phase-Uncertain Digital Receiver

This paper presents a Bayesian approach to symbol and phase inference in...
07/04/2011 ∙ by Arijit Das, et al. ∙ 0

• The variational Laplace approach to approximate Bayesian inference

Variational approaches to approximate Bayesian inference provide very ef...
03/02/2017 ∙ by Jean Daunizeau, et al. ∙ 0

• β-Cores: Robust Large-Scale Bayesian Data Summarization in the Presence of Outliers

Modern machine learning applications should be able to address the intri...
08/31/2020 ∙ by Dionysis Manousakas, et al. ∙ 9