Improved Algorithms for Agnostic Pool-based Active Classification

05/13/2021
by   Julian Katz-Samuels, et al.
0

We consider active learning for binary classification in the agnostic pool-based setting. The vast majority of works in active learning in the agnostic setting are inspired by the CAL algorithm where each query is uniformly sampled from the disagreement region of the current version space. The sample complexity of such algorithms is described by a quantity known as the disagreement coefficient which captures both the geometry of the hypothesis space as well as the underlying probability space. To date, the disagreement coefficient has been justified by minimax lower bounds only, leaving the door open for superior instance dependent sample complexities. In this work we propose an algorithm that, in contrast to uniform sampling over the disagreement region, solves an experimental design problem to determine a distribution over examples from which to request labels. We show that the new approach achieves sample complexity bounds that are never worse than the best disagreement coefficient-based bounds, but in specific cases can be dramatically smaller. From a practical perspective, the proposed algorithm requires no hyperparameters to tune (e.g., to control the aggressiveness of sampling), and is computationally efficient by means of assuming access to an empirical risk minimization oracle (without any constraints). Empirically, we demonstrate that our algorithm is superior to state of the art agnostic active learning algorithms on image classification datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2019

Active Learning for Binary Classification with Abstention

We construct and analyze active learning algorithms for the problem of b...
research
09/26/2013

Building Bridges: Viewing Active Learning from the Multi-Armed Bandit Lens

In this paper we propose a multi-armed bandit inspired, pool based activ...
research
06/29/2015

Efficient and Parsimonious Agnostic Active Learning

We develop a new active learning algorithm for the streaming setting sat...
research
11/09/2021

Practical, Provably-Correct Interactive Learning in the Realizable Setting: The Power of True Believers

We consider interactive learning in the realizable setting and develop a...
research
04/05/2014

A Compression Technique for Analyzing Disagreement-Based Active Learning

We introduce a new and improved characterization of the label complexity...
research
02/02/2016

Interactive algorithms: from pool to stream

We consider interactive algorithms in the pool-based setting, and in the...
research
12/15/2020

Generalized Chernoff Sampling for Active Learning and Structured Bandit Algorithms

Active learning and structured stochastic bandit problems are intimately...

Please sign up or login with your details

Forgot password? Click here to reset