Repulsion, Chaos and Equilibrium in Mixture Models

by   Andrea Cremaschi, et al.

Mixture models are commonly used in applications with heterogeneity and overdispersion in the population, as they allow the identification of subpopulations. In the Bayesian framework, this entails the specification of suitable prior distributions for the weights and location parameters of the mixture. Widely used are Bayesian semi-parametric models based on mixtures with infinite or random number of components, such as Dirichlet process mixtures or mixtures with random number of components. Key in this context is the choice of the kernel for cluster identification. Despite their popularity, the flexibility of these models and prior distributions often does not translate into interpretability of the identified clusters. To overcome this issue, clustering methods based on repulsive mixtures have been recently proposed. The basic idea is to include a repulsive term in the prior distribution of the atoms of the mixture, which favours mixture locations far apart. This approach is increasingly popular and allows one to produce well-separated clusters, thus facilitating the interpretation of the results. However, the resulting models are usually not easy to handle due to the introduction of unknown normalising constants. Exploiting results from statistical mechanics, we propose in this work a novel class of repulsive prior distributions based on Gibbs measures. Specifically, we use Gibbs measures associated to joint distributions of eigenvalues of random matrices, which naturally possess a repulsive property. The proposed framework greatly simplifies the computations needed for the use of repulsive mixtures due to the availability of the normalising constant in closed form. We investigate theoretical properties of such class of prior distributions, and illustrate the novel class of priors and their properties, as well as their clustering performance, on benchmark datasets.


page 1

page 2

page 3

page 4


Normalized Random Meaures with Interacting Atoms for Bayesian Nonparametric Mixtures

The study of almost surely discrete random probability measures is an ac...

Bayesian Analysis of Stochastic Volatility Model using Finite Gaussian Mixtures with Unknown Number of Components

Financial studies require volatility based models which provides useful ...

Gibbs sampling for mixtures in order of appearance: the ordered allocation sampler

Gibbs sampling methods for mixture models are based on data augmentation...

Spying on the prior of the number of data clusters and the partition distribution in Bayesian cluster analysis

Mixture models represent the key modelling approach for Bayesian cluster...

Smoothed Hierarchical Dirichlet Process: A Non-Parametric Approach to Constraint Measures

Time-varying mixture densities occur in many scenarios, for example, the...

Bayesian nonparametric Plackett-Luce models for the analysis of preferences for college degree programmes

In this paper we propose a Bayesian nonparametric model for clustering p...

Variance matrix priors for Dirichlet process mixture models with Gaussian kernels

The Dirichlet Process Mixture Model (DPMM) is a Bayesian non-parametric ...

Please sign up or login with your details

Forgot password? Click here to reset