DeepAI
Log In Sign Up

Improved Adaptive Algorithm for Scalable Active Learning with Weak Labeler

11/04/2022
by   Yifang Chen, et al.
0

Active learning with strong and weak labelers considers a practical setting where we have access to both costly but accurate strong labelers and inaccurate but cheap predictions provided by weak labelers. We study this problem in the streaming setting, where decisions must be taken online. We design a novel algorithmic template, Weak Labeler Active Cover (WL-AC), that is able to robustly leverage the lower quality weak labelers to reduce the query complexity while retaining the desired level of accuracy. Prior active learning algorithms with access to weak labelers learn a difference classifier which predicts where the weak labels differ from strong labelers; this requires the strong assumption of realizability of the difference classifier (Zhang and Chaudhuri,2015). WL-AC bypasses this realizability assumption and thus is applicable to many real-world scenarios such as random corrupted weak labels and high dimensional family of difference classifiers (e.g., deep neural nets). Moreover, WL-AC cleverly trades off evaluating the quality with full exploitation of weak labelers, which allows to convert any active learning strategy to one that can leverage weak labelers. We provide an instantiation of this template that achieves the optimal query complexity for any given weak labeler, without knowing its accuracy a-priori. Empirically, we propose an instantiation of the WL-AC template that can be efficiently implemented for large-scale models (e.g., deep neural nets) and show its effectiveness on the corrupted-MNIST dataset by significantly reducing the number of labels while keeping the same accuracy as in passive learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/09/2015

Active Learning from Weak and Strong Labelers

An active learner is given a hypothesis class, a large set of unlabeled ...
08/07/2019

An Adaptive Supervision Framework for Active Learning in Object Detection

Active learning approaches in computer vision generally involve querying...
10/06/2021

Active Learning for Sound Negotiations

We present two active learning algorithms for sound deterministic negoti...
04/30/2021

Active WeaSuL: Improving Weak Supervision with Active Learning

The availability of labelled data is one of the main limitations in mach...
10/30/2016

Active Learning from Imperfect Labelers

We study active learning where the labeler can not only return incorrect...
06/21/2021

Corruption Robust Active Learning

We conduct theoretical studies on streaming-based active learning for bi...
01/13/2017

Active Self-Paced Learning for Cost-Effective and Progressive Face Identification

This paper aims to develop a novel cost-effective framework for face ide...