Efficient active learning of sparse halfspaces with arbitrary bounded noise

02/12/2020
by   Chicheng Zhang, et al.
11

In this work we study active learning of homogeneous s-sparse halfspaces in R^d under label noise. Even in the absence of label noise this is a challenging problem and only recently have label complexity bounds of the form Õ(s ·polylog(d, 1/ϵ) ) been established in <cit.> for computationally efficient algorithms under the broad class of isotropic log-concave distributions. In contrast, under high levels of label noise, the label complexity bounds achieved by computationally efficient algorithms are much worse. When the label noise satisfies the Massart condition <cit.>, i.e., each label is flipped with probability at most η for a parameter η∈ [0,1/2), the work of <cit.> provides a computationally efficient active learning algorithm under isotropic log-concave distributions with label complexity Õ(s^poly(1/(1-2η))poly(log d, 1/ϵ) ). Hence the algorithm is label-efficient only when the noise rate η is a constant. In this work, we substantially improve on the state of the art by designing a polynomial time algorithm for active learning of s-sparse halfspaces under bounded noise and isotropic log-concave distributions, with a label complexity of Õ(s/(1-2η)^4polylog (d, 1/ϵ) ). Hence, our new algorithm is label-efficient even for noise rates close to 1/2. Prior to our work, such a result was not known even for the random classification noise model. Our algorithm builds upon existing margin-based algorithmic framework and at each iteration performs a sequence of online mirror descent updates on a carefully chosen loss sequence, and uses a novel gradient update rule that accounts for the bounded noise.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/31/2013

The Power of Localization for Efficiently Learning Linear Separators with Noise

We introduce a new approach for designing computationally efficient lear...
research
02/10/2021

Improved Algorithms for Efficient Active Learning Halfspaces with Massart and Tsybakov noise

We develop a computationally-efficient PAC active learning algorithm for...
research
12/19/2020

On the Power of Localized Perceptron for Label-Optimal Learning of Halfspaces with Adversarial Noise

We study online active learning of homogeneous halfspaces in ℝ^d with ad...
research
12/07/2011

Active Learning of Halfspaces under a Margin Assumption

We derive and analyze a new, efficient, pool-based active learning algor...
research
03/22/2017

S-Concave Distributions: Towards Broader Distributions for Noise-Tolerant and Sample-Efficient Learning Algorithms

We provide new results concerning noise-tolerant and sample-efficient le...
research
10/18/2020

Robust Learning under Strong Noise via SQs

This work provides several new insights on the robustness of Kearns' sta...
research
11/06/2012

Active and passive learning of linear separators under log-concave distributions

We provide new results concerning label efficient, polynomial time, pass...

Please sign up or login with your details

Forgot password? Click here to reset