Exact and approximate inference in graphical models: variable elimination and beyond

by   Nathalie Peyrard, et al.

Probabilistic graphical models offer a powerful framework to account for the dependence structure between variables, which can be represented as a graph. The dependence between variables may render inference tasks such as computing normalizing constant, marginalization or optimization intractable. The objective of this paper is to review techniques exploiting the graph structure for exact inference borrowed from optimization and computer science. They are not yet standard in the statistician toolkit, and we specify under which conditions they are efficient in practice. They are built on the principle of variable elimination whose complexity is dictated in an intricate way by the order in which variables are eliminated in the graph. The so-called treewidth of the graph characterizes this algorithmic complexity: low-treewidth graphs can be processed efficiently. Algorithmic solutions derived from variable elimination and the notion of treewidth are illustrated on problems of treewidth computation and inference in challenging benchmarks from optimization competitions. We also review how efficient techniques for approximate inference such as loopy belief propagation and variational approaches can be linked to variable elimination and we illustrate them in the context of Expectation-Maximisation procedures for parameter estimation in coupled Hidden Markov Models.


page 1

page 2

page 3

page 4


Complexity of Inference in Graphical Models

It is well-known that inference in graphical models is hard in the worst...

An Efficient Algorithm for Computing Interventional Distributions in Latent Variable Causal Models

Probabilistic inference in graphical models is the task of computing mar...

Approximation by Quantization

Inference in graphical models consists of repeatedly multiplying and sum...

Lifted Variable Elimination: Decoupling the Operators from the Constraint Language

Lifted probabilistic inference algorithms exploit regularities in the st...

An Advance on Variable Elimination with Applications to Tensor-Based Computation

We present new results on the classical algorithm of variable eliminatio...

CREPO: An Open Repository to Benchmark Credal Network Algorithms

Credal networks are a popular class of imprecise probabilistic graphical...

A Concise Function Representation for Faster Exact MPE and Constrained Optimisation in Graphical Models

We propose a novel concise function representation for graphical models,...

Please sign up or login with your details

Forgot password? Click here to reset