Specialists Outperform Generalists in Ensemble Classification

07/09/2021
by   Sascha Meyen, et al.
0

Consider an ensemble of k individual classifiers whose accuracies are known. Upon receiving a test point, each of the classifiers outputs a predicted label and a confidence in its prediction for this particular test point. In this paper, we address the question of whether we can determine the accuracy of the ensemble. Surprisingly, even when classifiers are combined in the statistically optimal way in this setting, the accuracy of the resulting ensemble classifier cannot be computed from the accuracies of the individual classifiers-as would be the case in the standard setting of confidence weighted majority voting. We prove tight upper and lower bounds on the ensemble accuracy. We explicitly construct the individual classifiers that attain the upper and lower bounds: specialists and generalists. Our theoretical results have very practical consequences: (1) If we use ensemble methods and have the choice to construct our individual (independent) classifiers from scratch, then we should aim for specialist classifiers rather than generalists. (2) Our bounds can be used to determine how many classifiers are at least required to achieve a desired ensemble accuracy. Finally, we improve our bounds by considering the mutual information between the true label and the individual classifier's output.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/16/2021

A Dataset-Level Geometric Framework for Ensemble Classifiers

Ensemble classifiers have been investigated by many in the artificial in...
research
09/09/2017

Less Is More: A Comprehensive Framework for the Number of Components of Ensemble Classifiers

The number of component classifiers chosen for an ensemble has a great i...
research
09/02/2022

On Effectively Predicting Autism Spectrum Disorder Using an Ensemble of Classifiers

An ensemble of classifiers combines several single classifiers to delive...
research
05/28/2021

Early Exiting with Ensemble Internal Classifiers

As a simple technique to accelerate inference of large-scale pre-trained...
research
08/27/2023

Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles

Ensembles, which employ a set of classifiers to enhance classification a...
research
10/02/2016

Sparsity-driven weighted ensemble classifier

In this letter, a novel weighted ensemble classifier is proposed that im...
research
07/29/2014

Estimating the Accuracies of Multiple Classifiers Without Labeled Data

In various situations one is given only the predictions of multiple clas...

Please sign up or login with your details

Forgot password? Click here to reset