How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learning

by   Sandra Gilhuber, et al.

Do we need active learning? The rise of strong deep semi-supervised methods raises doubt about the usability of active learning in limited labeled data settings. This is caused by results showing that combining semi-supervised learning (SSL) methods with a random selection for labeling can outperform existing active learning (AL) techniques. However, these results are obtained from experiments on well-established benchmark datasets that can overestimate the external validity. However, the literature lacks sufficient research on the performance of active semi-supervised learning methods in realistic data scenarios, leaving a notable gap in our understanding. Therefore we present three data challenges common in real-world applications: between-class imbalance, within-class imbalance, and between-class similarity. These challenges can hurt SSL performance due to confirmation bias. We conduct experiments with SSL and AL on simulated data challenges and find that random sampling does not mitigate confirmation bias and, in some cases, leads to worse performance than supervised learning. In contrast, we demonstrate that AL can overcome confirmation bias in SSL in these realistic settings. Our results provide insights into the potential of combining active and semi-supervised learning in the presence of common real-world challenges, which is a promising direction for robust methods when learning with limited labeled data in real-world applications.


page 1

page 2

page 3

page 4


Semantic Segmentation with Active Semi-Supervised Representation Learning

Obtaining human per-pixel labels for semantic segmentation is incredibly...

Consistency-Based Semi-supervised Evidential Active Learning for Diagnostic Radiograph Classification

Deep learning approaches achieve state-of-the-art performance for classi...

Collaborative Intelligence Orchestration: Inconsistency-Based Fusion of Semi-Supervised Learning and Active Learning

While annotating decent amounts of data to satisfy sophisticated learnin...

Semi-Supervised Active Learning for Support Vector Machines: A Novel Approach that Exploits Structure Information in Data

In our today's information society more and more data emerges, e.g. in s...

ED2: Two-stage Active Learning for Error Detection – Technical Report

Traditional error detection approaches require user-defined parameters a...

Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching

In real-world classification problems, the class balance in the training...

An Empirical Study on the Efficacy of Deep Active Learning for Image Classification

Deep Active Learning (DAL) has been advocated as a promising method to r...

Please sign up or login with your details

Forgot password? Click here to reset