Fast Stochastic Quadrature for Approximate Maximum-Likelihood Estimation

by   Katharina Morik, et al.

Recent stochastic quadrature techniques for undirected graphical models rely on near-minimax degree-k polynomial approximations to the model’s potential function for inferring the partition function. While providing desirable statistical guarantees, typical constructions of such approximations are themselves not amenable to efficient inference. Here, we develop a class of Monte Carlo sampling algorithms for efficiently approximating the value of the partition function, as well as the associated pseudo-marginals. More precisely, for pairwise models with n vertices and m edges, the complexity can be reduced from O(d^k) to O(k^4 + kn + m), where d ≥4 m is the parameter dimension. We also consider the uses of stochastic quadrature for the problem of maximum-likelihood (ML) parameter estimation. For completely observed data, our analysis gives rise to a probabilistic bound on the log-likelihood of the model. Maximizing this bound yields an approximate ML estimate which, in analogy to the moment-matching of exact ML estimation, can be interpreted in terms of pseudo-moment-matching. We present experimental results illustrating the behavior of this approximate ML estimator.


page 1

page 2

page 3

page 4


Comparative Analysis of Viterbi Training and Maximum Likelihood Estimation for HMMs

We present an asymptotic analysis of Viterbi Training (VT) and contrast ...

Maximum Likelihood Estimation of Toric Fano Varieties

We study the maximum likelihood estimation problem for several classes o...

BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees

The rising volume of datasets has made training machine learning (ML) mo...

What Cannot be Learned with Bethe Approximations

We address the problem of learning the parameters in graphical models wh...

Efficient Exact Inference in Planar Ising Models

We give polynomial-time algorithms for the exact computation of lowest-e...

Parameter Learning for Log-supermodular Distributions

We consider log-supermodular models on binary variables, which are proba...

Partition Function Estimation: A Quantitative Study

Probabilistic graphical models have emerged as a powerful modeling tool ...