DeepAI AI Chat
Log In Sign Up

Stopping criterion for active learning based on deterministic generalization bounds

by   Hideaki Ishibashi, et al.

Active learning is a framework in which the learning machine can select the samples to be used for training. This technique is promising, particularly when the cost of data acquisition and labeling is high. In active learning, determining the timing at which learning should be stopped is a critical issue. In this study, we propose a criterion for automatically stopping active learning. The proposed stopping criterion is based on the difference in the expected generalization errors and hypothesis testing. We derive a novel upper bound for the difference in expected generalization errors before and after obtaining a new training datum based on PAC-Bayesian theory. Unlike ordinary PAC-Bayesian bounds, though, the proposed bound is deterministic; hence, there is no uncontrollable trade-off between the confidence and tightness of the inequality. We combine the upper bound with a statistical test to derive a stopping criterion for active learning. We demonstrate the effectiveness of the proposed method via experiments with both artificial and real datasets.


page 1

page 2

page 3

page 4


Stopping Criterion for Active Learning Based on Error Stability

Active learning is a framework for supervised learning to improve the pr...

Diameter-Based Active Learning

To date, the tightest upper and lower-bounds for the active learning of ...

Multi-class Active Learning: A Hybrid Informative and Representative Criterion Inspired Approach

Labeling each instance in a large dataset is extremely labor- and time- ...

Analysis of Stopping Active Learning based on Stabilizing Predictions

Within the natural language processing (NLP) community, active learning ...

Hitting the Target: Stopping Active Learning at the Cost-Based Optimum

Active learning allows machine learning models to be trained using fewer...

New bounds for k-means and information k-means

In this paper, we derive a new dimension-free non-asymptotic upper bound...

Learning Time Dependent Choice

We explore questions dealing with the learnability of models of choice o...