ProBoost: a Boosting Method for Probabilistic Classifiers

09/04/2022
by   Fábio Mendonça, et al.
3

ProBoost, a new boosting algorithm for probabilistic classifiers, is proposed in this work. This algorithm uses the epistemic uncertainty of each training sample to determine the most challenging/uncertain ones; the relevance of these samples is then increased for the next weak learner, producing a sequence that progressively focuses on the samples found to have the highest uncertainty. In the end, the weak learners' outputs are combined into a weighted ensemble of classifiers. Three methods are proposed to manipulate the training set: undersampling, oversampling, and weighting the training samples according to the uncertainty estimated by the weak learners. Furthermore, two approaches are studied regarding the ensemble combination. The weak learner herein considered is a standard convolutional neural network, and the probabilistic models underlying the uncertainty estimation use either variational inference or Monte Carlo dropout. The experimental evaluation carried out on MNIST benchmark datasets shows that ProBoost yields a significant performance improvement. The results are further highlighted by assessing the relative achievable improvement, a metric proposed in this work, which shows that a model with only four weak learners leads to an improvement exceeding 12 either accuracy, sensitivity, or specificity), in comparison to the model learned without ProBoost.

READ FULL TEXT

page 5

page 8

page 13

page 16

page 17

page 18

page 19

page 20

research
06/25/2023

Language models are weak learners

A central notion in practical and theoretical machine learning is that o...
research
01/27/2023

AdaBoost is not an Optimal Weak to Strong Learner

AdaBoost is a classic boosting algorithm for combining multiple inaccura...
research
06/03/2022

Optimal Weak to Strong Learning

The classic algorithm AdaBoost allows to convert a weak learner, that is...
research
12/08/2014

Word learning under infinite uncertainty

Language learners must learn the meanings of many thousands of words, de...
research
01/23/2023

The Impossibility of Parallelizing Boosting

The aim of boosting is to convert a sequence of weak learners into a str...
research
04/26/2021

Exploring Bayesian Deep Learning for Urgent Instructor Intervention Need in MOOC Forums

Massive Open Online Courses (MOOCs) have become a popular choice for e-l...

Please sign up or login with your details

Forgot password? Click here to reset