DeepAI AI Chat
Log In Sign Up

Improving parameter learning of Bayesian nets from incomplete data

by   Giorgio Corani, et al.

This paper addresses the estimation of parameters of a Bayesian network from incomplete data. The task is usually tackled by running the Expectation-Maximization (EM) algorithm several times in order to obtain a high log-likelihood estimate. We argue that choosing the maximum log-likelihood estimate (as well as the maximum penalized log-likelihood and the maximum a posteriori estimate) has severe drawbacks, being affected both by overfitting and model uncertainty. Two ideas are discussed to overcome these issues: a maximum entropy approach and a Bayesian model averaging approach. Both ideas can be easily applied on top of EM, while the entropy idea can be also implemented in a more sophisticated way, through a dedicated non-linear solver. A vast set of experiments shows that these ideas produce significantly better estimates and inferences than the traditional and widely used maximum (penalized) log-likelihood and maximum a posteriori estimates. In particular, if EM is adopted as optimization engine, the model averaging approach is the best performing one; its performance is matched by the entropy approach when implemented using the non-linear solver. The results suggest that the applicability of these ideas is immediate (they are easy to implement and to integrate in currently available inference engines) and that they constitute a better way to learn Bayesian network parameters.


page 1

page 2

page 3

page 4


EM's Convergence in Gaussian Latent Tree Models

We study the optimization landscape of the log-likelihood function and t...

Benefits of over-parameterization with EM

Expectation Maximization (EM) is among the most popular algorithms for m...

Maximum a Posteriori Estimation of Dynamic Factor Models with Incomplete Data

In this paper, we present a method of maximum a posteriori estimation of...

A Projected Non-Linear Conjugate Gradient Algorithm for Destructive Negative Binomial Cure Rate Model

In this paper, we propose a new estimation methodology based on a projec...

Strong maximum a posteriori estimation in Banach spaces with Gaussian priors

We show that a nonparametric Bayesian inverse problem with a Gaussian pr...

Bayesian Conditional Gaussian Network Classifiers with Applications to Mass Spectra Classification

Classifiers based on probabilistic graphical models are very effective. ...

Model-based multi-parameter mapping

Quantitative MR imaging is increasingly favoured for its richer informat...