DeepAI AI Chat
Log In Sign Up

Condorcet's Jury Theorem for Consensus Clustering and its Implications for Diversity

by   Brijnesh J. Jain, et al.

Condorcet's Jury Theorem has been invoked for ensemble classifiers to indicate that the combination of many classifiers can have better predictive performance than a single classifier. Such a theoretical underpinning is unknown for consensus clustering. This article extends Condorcet's Jury Theorem to the mean partition approach under the additional assumptions that a unique ground-truth partition exists and sample partitions are drawn from a sufficiently small ball containing the ground-truth. As an implication of practical relevance, we question the claim that the quality of consensus clustering depends on the diversity of the sample partitions. Instead, we conjecture that limiting the diversity of the mean partitions is necessary for controlling the quality.


page 1

page 2

page 3

page 4


The Mean Partition Theorem of Consensus Clustering

To devise efficient solutions for approximating a mean partition in cons...

Asymptotic Behavior of Mean Partitions in Consensus Clustering

Although consistency is a minimum requirement of any estimator, little i...

Homogeneity of Cluster Ensembles

The expectation and the mean of partitions generated by a cluster ensemb...

Selective clustering ensemble based on kappa and F-score

Clustering ensemble has an impressive performance in improving the accur...

DivClust: Controlling Diversity in Deep Clustering

Clustering has been a major research topic in the field of machine learn...

Algebraic Ground Truth Inference: Non-Parametric Estimation of Sample Errors by AI Algorithms

Binary classification is widely used in ML production systems. Monitorin...

Robust Consensus Clustering and its Applications for Advertising Forecasting

Consensus clustering aggregates partitions in order to find a better fit...