The Label Complexity of Active Learning from Observational Data

by   Songbai Yan, et al.
University of California, San Diego

Counterfactual learning from observational data involves learning a classifier on an entire population based on data that is observed conditioned on a selection policy. This work considers this problem in an active setting, where the learner additionally has access to unlabeled examples and can choose to get a subset of these labeled by an oracle. Prior work on this problem uses disagreement-based active learning, along with an importance weighted loss estimator to account for counterfactuals, which leads to a high label complexity. We show how to instead incorporate a more efficient counterfactual risk minimizer into the active learning algorithm. This requires us to modify both the counterfactual risk to make it amenable to active learning, as well as the active learning process to make it amenable to the risk. We provably demonstrate that the result of this is an algorithm which is statistically consistent as well as more label-efficient than prior work.


page 1

page 2

page 3

page 4


Active Learning with Logged Data

We consider active learning with logged data, where labeled examples are...

Towards Efficient Active Learning of PDFA

We propose a new active learning algorithm for PDFA based on three main ...

Beating the Minimax Rate of Active Learning with Prior Knowledge

Active learning refers to the learning protocol where the learner is all...

Search Improves Label for Active Learning

We investigate active learning with access to two distinct oracles: Labe...

Efficient and Parsimonious Agnostic Active Learning

We develop a new active learning algorithm for the streaming setting sat...

Active Model Aggregation via Stochastic Mirror Descent

We consider the problem of learning convex aggregation of models, that i...

Active Covering

We analyze the problem of active covering, where the learner is given an...

Please sign up or login with your details

Forgot password? Click here to reset