List Online Classification

03/27/2023
by   Shay Moran, et al.
0

We study multiclass online prediction where the learner can predict using a list of multiple labels (as opposed to just one label in the traditional setting). We characterize learnability in this model using the b-ary Littlestone dimension. This dimension is a variation of the classical Littlestone dimension with the difference that binary mistake trees are replaced with (k+1)-ary mistake trees, where k is the number of labels in the list. In the agnostic setting, we explore different scenarios depending on whether the comparator class consists of single-labeled or multi-labeled functions and its tradeoff with the size of the lists the algorithm uses. We find that it is possible to achieve negative regret in some cases and provide a complete characterization of when this is possible. As part of our work, we adapt classical algorithms such as Littlestone's SOA and Rosenblatt's Perceptron to predict using lists of labels. We also establish combinatorial results for list-learnable classes, including an list online version of the Sauer-Shelah-Perles Lemma. We state our results within the framework of pattern classes – a generalization of hypothesis classes which can represent adaptive hypotheses (i.e. functions with memory), and model data-dependent assumptions such as linear classification with margin.

READ FULL TEXT
research
11/07/2022

A Characterization of List Learnability

A classical result in learning theory shows the equivalence of PAC learn...
research
06/25/2021

Littlestone Classes are Privately Online Learnable

We consider the problem of online classification under a privacy constra...
research
11/16/2022

Comparative Learning: A Sample Complexity Theory for Two Hypothesis Classes

In many learning theory problems, a central role is played by a hypothes...
research
03/03/2022

A Characterization of Multiclass Learnability

A seminal result in learning theory characterizes the PAC learnability o...
research
06/09/2023

Online Learning with Set-Valued Feedback

We study a variant of online multiclass classification where the learner...
research
01/06/2023

A Characterization of Multilabel Learnability

We consider the problem of multilabel classification and investigate lea...
research
01/10/2018

Buying Online - A Characterization of Rational Buying Procedures

In decision theory, an agent chooses from a set of alternatives. When bu...

Please sign up or login with your details

Forgot password? Click here to reset