Optimal SQ Lower Bounds for Learning Halfspaces with Massart Noise

01/24/2022
by   Rajai Nasser, et al.
0

We give tight statistical query (SQ) lower bounds for learnining halfspaces in the presence of Massart noise. In particular, suppose that all labels are corrupted with probability at most η. We show that for arbitrary η∈ [0,1/2] every SQ algorithm achieving misclassification error better than η requires queries of superpolynomial accuracy or at least a superpolynomial number of queries. Further, this continues to hold even if the information-theoretically optimal error OPT is as small as exp(-log^c(d)), where d is the dimension and 0 < c < 1 is an arbitrary absolute constant, and an overwhelming fraction of examples are noiseless. Our lower bound matches known polynomial time algorithms, which are also implementable in the SQ framework. Previously, such lower bounds only ruled out algorithms achieving error OPT + ϵ or error better than Ω(η) or, if η is close to 1/2, error η - o_η(1), where the term o_η(1) is constant in d but going to 0 for η approaching 1/2. As a consequence, we also show that achieving misclassification error better than 1/2 in the (A,α)-Tsybakov model is SQ-hard for A constant and α bounded away from 1.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/17/2020

Hardness of Learning Halfspaces with Massart Noise

We study the complexity of PAC learning halfspaces in the presence of Ma...
research
02/10/2022

Near-Optimal Statistical Query Lower Bounds for Agnostically Learning Intersections of Halfspaces with Gaussian Marginals

We consider the well-studied problem of learning intersections of halfsp...
research
05/28/2018

High Probability Frequency Moment Sketches

We consider the problem of sketching the p-th frequency moment of a vect...
research
06/02/2023

Efficient Quantum State Synthesis with One Query

We present a polynomial-time quantum algorithm making a single query (in...
research
11/02/2019

Order Optimal One-Shot Distributed Learning

We consider distributed statistical optimization in one-shot setting, wh...
research
04/08/2016

Challenges in Bayesian Adaptive Data Analysis

Traditional statistical analysis requires that the analysis process and ...
research
06/22/2022

The Bounded Beam Search algorithm for the Block Relocation Problem

In this paper we deal with the restricted Block Relocation Problem. We p...

Please sign up or login with your details

Forgot password? Click here to reset