Time-Constrained Learning

02/04/2022
by   Sergio Filho, et al.
0

Consider a scenario in which we have a huge labeled dataset D and a limited time to train some given learner using D. Since we may not be able to use the whole dataset, how should we proceed? Questions of this nature motivate the definition of the Time-Constrained Learning Task (TCL): Given a dataset D sampled from an unknown distribution μ, a learner L and a time limit T, the goal is to obtain in at most T units of time the classification model with highest possible accuracy w.r.t. to μ, among those that can be built by L using the dataset D. We propose TCT, an algorithm for the TCL task designed based that on principles from Machine Teaching. We present an experimental study involving 5 different Learners and 20 datasets where we show that TCT consistently outperforms two other algorithms: the first is a Teacher for black-box learners proposed in [Dasgupta et al., ICML 19] and the second is a natural adaptation of random sampling for the TCL setting. We also compare TCT with Stochastic Gradient Descent training – our method is again consistently better. While our work is primarily practical, we also show that a stripped-down version of TCT has provable guarantees. Under reasonable assumptions, the time our algorithm takes to achieve a certain accuracy is never much bigger than the time it takes the batch teacher (which sends a single batch of examples) to achieve similar accuracy, and in some case it is almost exponentially better.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/30/2017

Iterative Machine Teaching

In this paper, we consider the problem of machine teaching, the inverse ...
research
10/21/2017

Towards Black-box Iterative Machine Teaching

In this paper, we make an important step towards the black-box machine t...
research
02/14/2018

Understanding the Role of Adaptivity in Machine Teaching: The Case of Version Space Learners

In real-world applications of education and human teaching, an effective...
research
06/30/2022

When an Active Learner Meets a Black-box Teacher

Active learning maximizes the hypothesis updates to find those desired u...
research
02/25/2018

Teacher Improves Learning by Selecting a Training Subset

We call a learner super-teachable if a teacher can trim down an iid trai...
research
06/17/2021

On Anytime Learning at Macroscale

Classical machine learning frameworks assume access to a possibly large ...
research
01/27/2020

Adaptive Teaching of Temporal Logic Formulas to Learners with Preferences

Machine teaching is an algorithmic framework for teaching a target hypot...

Please sign up or login with your details

Forgot password? Click here to reset