Adjusting for Chance Clustering Comparison Measures

12/03/2015
by   Simone Romano, et al.
0

Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI) based on Shannon information theory are very popular in the clustering community. Nonetheless it is an open problem as to what are the best application scenarios for each measure and guidelines in the literature for their usage are sparse, with the result that users often resort to using both. Generalized Information Theoretic (IT) measures based on the Tsallis entropy have been shown to link pair-counting and Shannon IT measures. In this paper, we aim to bridge the gap between adjustment of measures based on pair-counting and measures based on information theory. We solve the key technical challenge of analytically computing the expected value and variance of generalized IT measures. This allows us to propose adjustments of generalized IT measures, which reduce to well known adjusted clustering comparison measures as special cases. Using the theory of generalized IT measures, we are able to propose the following guidelines for using ARI and AMI as external validation indices: ARI should be used when the reference clustering has large equal sized clusters; AMI should be used when the reference clustering is unbalanced and there exist small clusters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/07/2019

Understanding partition comparison indices based on counting object pairs

In unsupervised machine learning, agreement between partitions is common...
research
11/17/2020

Adjusting the adjusted Rand Index – A multinomial story

The Adjusted Rand Index (ARI) is arguably one of the most popular measur...
research
09/07/2022

Adjusted Asymmetric Accuracy: A Well-Behaving External Cluster Validity Measure

There is no, nor will there ever be, single best clustering algorithm, b...
research
07/26/2019

A close-up comparison of the misclassification error distance and the adjusted Rand index for external clustering evaluation

The misclassification error distance and the adjusted Rand index are two...
research
05/03/2023

FastAMI – a Monte Carlo Approach to the Adjustment for Chance in Clustering Comparison Metrics

Clustering is at the very core of machine learning, and its applications...
research
11/12/2019

Systematic Analysis of Cluster Similarity Indices: Towards Bias-free Cluster Validation

There are many cluster similarity indices used to evaluate clustering al...
research
07/16/2018

On the Information Theoretic Distance Measures and Bidirectional Helmholtz Machines

By establishing a connection between bi-directional Helmholtz machines a...

Please sign up or login with your details

Forgot password? Click here to reset