Convergent message passing algorithms - a unifying view

by   Talya Meltzer, et al.

Message-passing algorithms have emerged as powerful techniques for approximate inference in graphical models. When these algorithms converge, they can be shown to find local (or sometimes even global) optima of variational formulations to the inference problem. But many of the most popular algorithms are not guaranteed to converge. This has lead to recent interest in convergent message-passing algorithms. In this paper, we present a unified view of convergent message-passing algorithms. We present a simple derivation of an abstract algorithm, tree-consistency bound optimization (TCBO) that is provably convergent in both its sum and max product forms. We then show that many of the existing convergent algorithms are instances of our TCBO algorithm, and obtain novel convergent algorithms "for free" by exchanging maximizations and summations in existing algorithms. In particular, we show that Wainwright's non-convergent sum-product algorithm for tree based variational bounds, is actually convergent with the right update order for the case where trees are monotonic chains.



There are no comments yet.


page 1

page 2

page 3

page 4


Message-Passing Algorithms: Reparameterizations and Splittings

The max-product algorithm, a local message-passing scheme that attempts ...

A new look at reweighted message passing

We propose a new family of message passing techniques for MAP estimation...

Faster Algorithms for Max-Product Message-Passing

Maximum A Posteriori inference in graphical models is often solved via m...

Convex Combination Belief Propagation Algorithms

We introduce new message passing algorithms for inference with graphical...

Robust Probabilistic Inference in Distributed Systems

Probabilistic inference problems arise naturally in distributed systems ...

TRAMP: Compositional Inference with TRee Approximate Message Passing

We introduce tramp, standing for TRee Approximate Message Passing, a pyt...

Realising Active Inference in Variational Message Passing: the Outcome-blind Certainty Seeker

Active inference is a state-of-the-art framework in neuroscience that of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.