Fast Convergence of Belief Propagation to Global Optima: Beyond Correlation Decay

05/24/2019
by   Frederic Koehler, et al.
0

Belief propagation is a fundamental message-passing algorithm for probabilistic reasoning and inference in graphical models. While it is known to be exact on trees, in most applications belief propagation is run on graphs with cycles. Understanding the behavior of "loopy" belief propagation has been a major challenge for researchers in machine learning, and positive convergence results for BP are known under strong assumptions which imply the underlying graphical model exhibits decay of correlations. We show that under a natural initialization, BP converges quickly to the global optimum of the Bethe free energy for Ising models on arbitrary graphs, as long as the Ising model is ferromagnetic (i.e. neighbors prefer to be aligned). This holds even though such models can exhibit long range correlations and may have multiple suboptimal BP fixed points. We also show an analogous result for iterating the (naive) mean-field equations; perhaps surprisingly, both results are dimension-free in the sense that a constant number of iterations already provides a good estimate to the Bethe/mean-field free energy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/11/2021

Convergence of Generalized Belief Propagation Algorithm on Graphs with Motifs

Belief propagation is a fundamental message-passing algorithm for numero...
research
06/17/2020

Region-based Energy Neural Network for Approximate Inference

Region-based free energy was originally proposed for generalized belief ...
research
12/02/2011

Merging Belief Propagation and the Mean Field Approximation: A Free Energy Approach

We present a joint message passing approach that combines belief propaga...
research
06/27/2014

Linearized and Single-Pass Belief Propagation

How can we tell when accounts are fake or real in a social network? And ...
research
11/04/2011

Stochastic Belief Propagation: A Low-Complexity Alternative to the Sum-Product Algorithm

The sum-product or belief propagation (BP) algorithm is a widely-used me...
research
07/27/2017

Anytime Exact Belief Propagation

Statistical Relational Models and, more recently, Probabilistic Programm...
research
09/24/2020

Message-Passing Algorithms and Homology

This PhD thesis lays out algebraic and topological structures relevant f...

Please sign up or login with your details

Forgot password? Click here to reset