Learning Geometric Concepts with Nasty Noise

07/05/2017
by   Ilias Diakonikolas, et al.
0

We study the efficient learnability of geometric concept classes - specifically, low-degree polynomial threshold functions (PTFs) and intersections of halfspaces - when a fraction of the data is adversarially corrupted. We give the first polynomial-time PAC learning algorithms for these concept classes with dimension-independent error guarantees in the presence of nasty noise under the Gaussian distribution. In the nasty noise model, an omniscient adversary can arbitrarily corrupt a small fraction of both the unlabeled data points and their labels. This model generalizes well-studied noise models, including the malicious noise model and the agnostic (adversarial label noise) model. Prior to our work, the only concept class for which efficient malicious learning algorithms were known was the class of origin-centered halfspaces. Specifically, our robust learning algorithm for low-degree PTFs succeeds under a number of tame distributions -- including the Gaussian distribution and, more generally, any log-concave distribution with (approximately) known low-degree moments. For LTFs under the Gaussian distribution, we give a polynomial-time algorithm that achieves error O(ϵ), where ϵ is the noise rate. At the core of our PAC learning results is an efficient algorithm to approximate the low-degree Chow-parameters of any bounded function in the presence of nasty noise. To achieve this, we employ an iterative spectral method for outlier detection and removal, inspired by recent work in robust unsupervised learning. Our aforementioned algorithm succeeds for a range of distributions satisfying mild concentration bounds and moment assumptions. The correctness of our robust learning algorithm for intersections of halfspaces makes essential use of a novel robust inverse independence lemma that may be of broader interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/31/2013

The Power of Localization for Efficiently Learning Linear Separators with Noise

We introduce a new approach for designing computationally efficient lear...
research
06/01/2023

Attribute-Efficient PAC Learning of Low-Degree Polynomial Threshold Functions with Nasty Noise

The concept class of low-degree polynomial threshold functions (PTFs) pl...
research
10/18/2020

Robust Learning under Strong Noise via SQs

This work provides several new insights on the robustness of Kearns' sta...
research
10/04/2020

A Polynomial Time Algorithm for Learning Halfspaces with Tsybakov Noise

We study the problem of PAC learning homogeneous halfspaces in the prese...
research
04/24/2016

Agnostic Estimation of Mean and Covariance

We consider the problem of estimating the mean and covariance of a distr...
research
03/08/2018

Efficient Algorithms for Outlier-Robust Regression

We give the first polynomial-time algorithm for performing linear or pol...
research
05/17/2023

Reaching Kesten-Stigum Threshold in the Stochastic Block Model under Node Corruptions

We study robust community detection in the context of node-corrupted sto...

Please sign up or login with your details

Forgot password? Click here to reset