Exact log-likelihood for clustering parameterised models and normally distributed data
share
Taking a model with equal means in each cluster, the log-likelihood for clustering multivariate normal distributions is calculated. The result has terms to penalise poor fits and model complexity, and determines both the number and composition of clusters. The procedure is equivalent to exactly calculating the Bayesian Information Criterion (BIC), and can produce similar, but less subjective results as the ad-hoc "elbow criterion". An intended application is clustering of fitted models, whose maximum likelihood estimates (MLEs) are normally distributed. Fitted models are often more familiar and interpretable than directly clustered data, can build-in prior knowledge, adjust for known confounders, and can use marginalisation to emphasise parameters of interest. That overall approach is equivalent to a multi-layer clustering algorithm that characterises features through the normally distributed MLE parameters of a fitted model, and then clusters the normal distributions. Alternatively, the results can be applied directly to the means and covariances of (possibly labelled) data.
READ FULL TEXT