Towards a combinatorial characterization of bounded memory learning

02/08/2020
by   Alon Gonen, et al.
0

Combinatorial dimensions play an important role in the theory of machine learning. For example, VC dimension characterizes PAC learning, SQ dimension characterizes weak learning with statistical queries, and Littlestone dimension characterizes online learning. In this paper we aim to develop combinatorial dimensions that characterize bounded memory learning. We propose a candidate solution for the case of realizable strong learning under a known distribution, based on the SQ dimension of neighboring distributions. We prove both upper and lower bounds for our candidate solution, that match in some regime of parameters. In this parameter regime there is an equivalence between bounded memory and SQ learning. We conjecture that our characterization holds in a much wider regime of parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2023

A Combinatorial Characterization of Online Learning Games with Bounded Losses

We study the online learnability of hypothesis classes with respect to a...
research
07/07/2023

Optimal Learners for Realizable Regression: PAC Learning and Online Learning

In this work, we aim to characterize the statistical complexity of reali...
research
12/10/2017

A General Memory-Bounded Learning Algorithm

In an era of big data there is a growing need for memory-bounded learnin...
research
08/23/2021

Primal and Dual Combinatorial Dimensions

We give tight bounds on the relation between the primal and dual of vari...
research
06/02/2020

A combinatorial conjecture from PAC-Bayesian machine learning

We present a proof of a combinatorial conjecture from the second author'...
research
06/09/2023

A Unified Model and Dimension for Interactive Estimation

We study an abstract framework for interactive learning called interacti...
research
04/16/2021

A Further Study of Quadratic APN Permutations in Dimension Nine

Recently, Beierle and Leander found two new sporadic quadratic APN permu...

Please sign up or login with your details

Forgot password? Click here to reset