Active Learning with Logged Data

by   Songbai Yan, et al.

We consider active learning with logged data, where labeled examples are drawn conditioned on a predetermined logging policy, and the goal is to learn a classifier on the entire population, not just conditioned on the logging policy. Prior work addresses this problem either when only logged data is available, or purely in a controlled random experimentation setting where the logged data is ignored. In this work, we combine both approaches to provide an algorithm that uses logged data to bootstrap and inform experimentation, thus achieving the best of both worlds. Our work is inspired by a connection between controlled random experimentation and active learning, and modifies existing disagreement-based active learning algorithms to exploit logged data.


page 1

page 2

page 3

page 4


The Label Complexity of Active Learning from Observational Data

Counterfactual learning from observational data involves learning a clas...

Learning a Policy for Opportunistic Active Learning

Active learning identifies data points to label that are expected to be ...

Batch Active Learning via Coordinated Matching

Most prior work on active learning of classifiers has focused on sequent...

The Practical Challenges of Active Learning: Lessons Learned from Live Experimentation

We tested in a live setting the use of active learning for selecting tex...

Dialog Policy Learning for Joint Clarification and Active Learning Queries

Intelligent systems need to be able to recover from mistakes, resolve un...

Investigating Multi-source Active Learning for Natural Language Inference

In recent years, active learning has been successfully applied to an arr...

Please sign up or login with your details

Forgot password? Click here to reset