Minimum Message Length Clustering Using Gibbs Sampling

01/16/2013
by   Ian Davidson, et al.
0

The K-Mean and EM algorithms are popular in clustering and mixture modeling, due to their simplicity and ease of implementation. However, they have several significant limitations. Both coverage to a local optimum of their respective objective functions (ignoring the uncertainty in the model space), require the apriori specification of the number of classes/clsuters, and are inconsistent. In this work we overcome these limitations by using the Minimum Message Length (MML) principle and a variation to the K-Means/EM observation assignment and parameter calculation scheme. We maintain the simplicity of these approaches while constructing a Bayesian mixture modeling tool that samples/searches the model space using a Markov Chain Monte Carlo (MCMC) sampler known as a Gibbs sampler. Gibbs sampling allows us to visit each model according to its posterior probability. Therefore, if the model space is multi-modal we will visit all models and not get stuck in local optima. We call our approach multiple chains at equilibrium (MCE) MML sampling.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8

research
09/25/2020

Multilevel Gibbs Sampling for Bayesian Regression

Bayesian regression remains a simple but effective tool based on Bayesia...
research
04/14/2020

Particle Gibbs Sampling for Bayesian Phylogenetic inference

The combinatorial sequential Monte Carlo (CSMC) has been demonstrated to...
research
12/16/2014

Testing MCMC code

Markov Chain Monte Carlo (MCMC) algorithms are a workhorse of probabilis...
research
06/05/2023

Gibbs Sampling the Posterior of Neural Networks

In this paper, we study sampling from a posterior derived from a neural ...
research
01/25/2020

Particle-Gibbs Sampling For Bayesian Feature Allocation Models

Bayesian feature allocation models are a popular tool for modelling data...
research
07/04/2018

Discrete Sampling using Semigradient-based Product Mixtures

We consider the problem of inference in discrete probabilistic models, t...
research
10/02/2014

Mapping Energy Landscapes of Non-Convex Learning Problems

In many statistical learning problems, the target functions to be optimi...

Please sign up or login with your details

Forgot password? Click here to reset