Generalized Ambiguity Decomposition for Understanding Ensemble Diversity

12/28/2013
by   Kartik Audhkhasi, et al.
0

Diversity or complementarity of experts in ensemble pattern recognition and information processing systems is widely-observed by researchers to be crucial for achieving performance improvement upon fusion. Understanding this link between ensemble diversity and fusion performance is thus an important research question. However, prior works have theoretically characterized ensemble diversity and have linked it with ensemble performance in very restricted settings. We present a generalized ambiguity decomposition (GAD) theorem as a broad framework for answering these questions. The GAD theorem applies to a generic convex ensemble of experts for any arbitrary twice-differentiable loss function. It shows that the ensemble performance approximately decomposes into a difference of the average expert performance and the diversity of the ensemble. It thus provides a theoretical explanation for the empirically-observed benefit of fusing outputs from diverse classifiers and regressors. It also provides a loss function-dependent, ensemble-dependent, and data-dependent definition of diversity. We present extensions of this decomposition to common regression and classification loss functions, and report a simulation-based analysis of the diversity term and the accuracy of the decomposition. We finally present experiments on standard pattern recognition data sets which indicate the accuracy of the decomposition for real-world classification and regression problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/10/2023

A Unified Theory of Diversity in Ensemble Learning

We present a theory of ensemble diversity, explaining the nature and eff...
research
11/05/2020

Generalized Negative Correlation Learning for Deep Ensembling

Ensemble algorithms offer state of the art performance in many machine l...
research
06/04/2014

Learning to Diversify via Weighted Kernels for Classifier Ensemble

Classifier ensemble generally should combine diverse component classifie...
research
03/11/2019

Multi-Representational Learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Offline Signature Verification (OSV) is a challenging pattern recognitio...
research
02/22/2019

Diversity of Ensembles for Data Stream Classification

When constructing a classifier ensemble, diversity among the base classi...
research
08/21/2023

DynED: Dynamic Ensemble Diversification in Data Stream Classification

Ensemble methods are commonly used in classification due to their remark...
research
08/15/2019

Deep learning for Plankton and Coral Classification

Oceans are the essential lifeblood of the Earth: they provide over 70 ox...

Please sign up or login with your details

Forgot password? Click here to reset