Pairwise MRF Calibration by Perturbation of the Bethe Reference Point

by   Cyril Furtlehner, et al.

We investigate different ways of generating approximate solutions to the pairwise Markov random field (MRF) selection problem. We focus mainly on the inverse Ising problem, but discuss also the somewhat related inverse Gaussian problem because both types of MRF are suitable for inference tasks with the belief propagation algorithm (BP) under certain conditions. Our approach consists in to take a Bethe mean-field solution obtained with a maximum spanning tree (MST) of pairwise mutual information, referred to as the Bethe reference point, for further perturbation procedures. We consider three different ways following this idea: in the first one, we select and calibrate iteratively the optimal links to be added starting from the Bethe reference point; the second one is based on the observation that the natural gradient can be computed analytically at the Bethe point; in the third one, assuming no local field and using low temperature expansion we develop a dual loop joint model based on a well chosen fundamental cycle basis. We indeed identify a subclass of planar models, which we refer to as Bethe-dual graph models, having possibly many loops, but characterized by a singly connected dual factor graph, for which the partition function and the linear response can be computed exactly in respectively O(N) and O(N^2) operations, thanks to a dual weight propagation (DWP) message passing procedure that we set up. When restricted to this subclass of models, the inverse Ising problem being convex, becomes tractable at any temperature. Experimental tests on various datasets with refined L_0 or L_1 regularization procedures indicate that these approaches may be competitive and useful alternatives to existing ones.


page 1

page 2

page 3

page 4


Characterizing and Improving Generalized Belief Propagation Algorithms on the 2D Edwards-Anderson Model

We study the performance of different message passing algorithms in the ...

Merging Belief Propagation and the Mean Field Approximation: A Free Energy Approach

We present a joint message passing approach that combines belief propaga...

Convergence Analysis of Distributed Inference with Vector-Valued Gaussian Belief Propagation

This paper considers inference over distributed linear Gaussian models u...

Scalable Variational Inference in Log-supermodular Models

We consider the problem of approximate Bayesian inference in log-supermo...

Diffusion Methods for Classification with Pairwise Relationships

We define two algorithms for propagating information in classification p...

Negative Tree Reweighted Belief Propagation

We introduce a new class of lower bounds on the log partition function o...

Marginal Densities, Factor Graph Duality, and High-Temperature Series Expansions

We prove that the marginals densities of a primal normal factor graph an...

Please sign up or login with your details

Forgot password? Click here to reset