Hardness of parameter estimation in graphical models

09/12/2014
by   Guy Bresler, et al.
0

We consider the problem of learning the canonical parameters specifying an undirected graphical model (Markov random field) from the mean parameters. For graphical models representing a minimal exponential family, the canonical parameters are uniquely determined by the mean parameters, so the problem is feasible in principle. The goal of this paper is to investigate the computational feasibility of this statistical task. Our main result shows that parameter estimation is in general intractable: no algorithm can learn the canonical parameters of a generic pair-wise binary graphical model from the mean parameters in time bounded by a polynomial in the number of variables (unless RP = NP). Indeed, such a result has been believed to be true (see the monograph by Wainwright and Jordan (2008)) but no proof was known. Our proof gives a polynomial time reduction from approximating the partition function of the hard-core model, known to be hard, to learning approximate parameters. Our reduction entails showing that the marginal polytope boundary has an inherent repulsive property, which validates an optimization procedure over the polytope that does not use any knowledge of its structure (as required by the ellipsoid method and others).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/04/2012

Learning Factor Graphs in Polynomial Time & Sample Complexity

We study computational and sample complexity of parameter and structure ...
research
01/22/2019

Lower bounds for testing graphical models: colorings and antiferromagnetic Ising models

We study the identity testing problem in the context of spin systems or ...
research
10/26/2020

Stochastic Discrete Clenshaw-Curtis Quadrature

The partition function is fundamental for probabilistic graphical models...
research
10/28/2014

Learning graphical models from the Glauber dynamics

In this paper we consider the problem of learning undirected graphical m...
research
05/01/2018

On the Equivalence of Generative and Discriminative Formulations of the Sequential Dependence Model

The sequential dependence model (SDM) is a popular retrieval model which...
research
03/14/2012

Bayesian Parameter Estimation for Latent Markov Random Fields and Social Networks

Undirected graphical models are widely used in statistics, physics and m...
research
07/11/2012

PAC-learning bounded tree-width Graphical Models

We show that the class of strongly connected graphical models with treew...

Please sign up or login with your details

Forgot password? Click here to reset