Learning active learning at the crossroads? evaluation and discussion

12/16/2020
by   Louis Desreumaux, et al.
0

Active learning aims to reduce annotation cost by predicting which samples are useful for a human expert to label. Although this field is quite old, several important challenges to using active learning in real-world settings still remain unsolved. In particular, most selection strategies are hand-designed, and it has become clear that there is no best active learning strategy that consistently outperforms all others in all applications. This has motivated research into meta-learning algorithms for "learning how to actively learn". In this paper, we compare this kind of approach with the association of a Random Forest with the margin sampling strategy, reported in recent comparative studies as a very competitive heuristic. To this end, we present the results of a benchmark performed on 20 datasets that compares a strategy learned using a recent meta-learning algorithm with margin sampling. We also present some lessons learned and open future perspectives.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/14/2023

Algorithm Selection for Deep Active Learning with Imbalanced Datasets

Label efficiency has become an increasingly important objective in deep ...
research
06/12/2018

Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning

Active learning (AL) aims to enable training high performance classifier...
research
09/29/2018

Dynamic Ensemble Active Learning: A Non-Stationary Bandit with Expert Advice

Active learning aims to reduce annotation cost by predicting which sampl...
research
11/29/2020

Active Output Selection Strategies for Multiple Learning Regression Models

Active learning shows promise to decrease test bench time for model-base...
research
06/13/2023

A Markovian Formalism for Active Querying

Active learning algorithms have been an integral part of recent advances...
research
07/07/2020

Meta-active Learning in Probabilistically-Safe Optimization

Learning to control a safety-critical system with latent dynamics (e.g. ...
research
06/16/2023

ActiveGLAE: A Benchmark for Deep Active Learning with Transformers

Deep active learning (DAL) seeks to reduce annotation costs by enabling ...

Please sign up or login with your details

Forgot password? Click here to reset