Extractor-Based Time-Space Lower Bounds for Learning

08/08/2017
by   Sumegha Garg, et al.
0

A matrix M: A × X →{-1,1} corresponds to the following learning problem: An unknown element x ∈ X is chosen uniformly at random. A learner tries to learn x from a stream of samples, (a_1, b_1), (a_2, b_2) ..., where for every i, a_i ∈ A is chosen uniformly at random and b_i = M(a_i,x). Assume that k,ℓ, r are such that any submatrix of M of at least 2^-k· |A| rows and at least 2^-ℓ· |X| columns, has a bias of at most 2^-r. We show that any learning algorithm for the learning problem corresponding to M requires either a memory of size at least Ω(k ·ℓ), or at least 2^Ω(r) samples. The result holds even if the learner has an exponentially small success probability (of 2^-Ω(r)). In particular, this shows that for a large class of learning problems, any learning algorithm requires either a memory of size at least Ω(( |X|) · ( |A|)) or an exponential number of samples, achieving a tight Ω(( |X|) · ( |A|)) lower bound on the size of the memory, rather than a bound of Ω({( |X|)^2,( |A|)^2}) obtained in previous works [R17,MM17b]. Moreover, our result implies all previous memory-samples lower bounds, as well as a number of new applications. Our proof builds on [R17] that gave a general technique for proving memory-samples lower bounds.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/05/2021

Memory-Sample Lower Bounds for Learning Parity with Noise

In this work, we show, for the well-studied problem of learning parity u...
research
02/17/2020

Time-Space Tradeoffs for Distinguishing Distributions and Applications to Security of Goldreich's PRG

In this work, we establish lower-bounds against memory bounded algorithm...
research
11/20/2017

On estimating the alphabet size of a discrete random source

We are concerned with estimating alphabet size N from a stream of symbol...
research
03/01/2023

Memory-Sample Lower Bounds for Learning with Classical-Quantum Hybrid Memory

In a work by Raz (J. ACM and FOCS 16), it was proved that any algorithm ...
research
08/08/2017

Time-Space Tradeoffs for Learning from Small Test Spaces: Learning Low Degree Polynomial Functions

We develop an extension of recently developed methods for obtaining time...
research
01/13/2023

Streaming Lower Bounds and Asymmetric Set-Disjointness

Frequency estimation in data streams is one of the classical problems in...
research
04/18/2019

Memory-Sample Tradeoffs for Linear Regression with Small Error

We consider the problem of performing linear regression over a stream of...

Please sign up or login with your details

Forgot password? Click here to reset