QActor: On-line Active Learning for Noisy Labeled Stream Data

01/28/2020
by   Taraneh Younesian, et al.
0

Noisy labeled data is more a norm than a rarity for self-generated content that is continuously published on the web and social media. Due to privacy concerns and governmental regulations, such a data stream can only be stored and used for learning purposes in a limited duration. To overcome the noise in this on-line scenario we propose QActor which novel combines: the selection of supposedly clean samples via quality models and actively querying an oracle for the most informative true labels. While the former can suffer from low data volumes of on-line scenarios, the latter is constrained by the availability and costs of human experts. QActor swiftly combines the merits of quality models for data filtering and oracle queries for cleaning the most informative data. The objective of QActor is to leverage the stringent oracle budget to robustly maximize the learning accuracy. QActor explores various strategies combining different query allocations and uncertainty measures. A central feature of QActor is to dynamically adjust the query limit according to the learning loss for each data batch. We extensively evaluate different image datasets fed into the classifier that can be standard machine learning (ML) models or deep neural networks (DNN) with noise label ratios ranging between 30 show that QActor can nearly match the optimal accuracy achieved using only clean data at the cost of at most an additional 6 the oracle.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2020

Active Learning for Noisy Data Streams Using Weak and Strong Labelers

Labeling data correctly is an expensive and challenging task in machine ...
research
06/17/2019

Active Generative Adversarial Network for Image Classification

Sufficient supervised information is crucial for any machine learning mo...
research
03/28/2019

Handling Noisy Labels for Robustly Learning from Self-Training Data for Low-Resource Sequence Labeling

In this paper, we address the problem of effectively self-training neura...
research
02/05/2020

Rényi Entropy Bounds on the Active Learning Cost-Performance Tradeoff

Semi-supervised classification, one of the most prominent fields in mach...
research
04/22/2020

SoQal: Selective Oracle Questioning in Active Learning

Large sets of unlabelled data within the healthcare domain remain underu...
research
10/05/2019

On the Limits of Learning to Actively Learn Semantic Representations

One of the goals of natural language understanding is to develop models ...

Please sign up or login with your details

Forgot password? Click here to reset