Revisiting Perceptron: Efficient and Label-Optimal Learning of Halfspaces
share
It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the bounded noise condition MN06, where each label is flipped with probability at most η < 1/2, our algorithm achieves a near-optimal label complexity of Õ(d/(1-2η)^21/ϵ) in time Õ(d^2/ϵ(1-2η)^3). Under the adversarial noise condition ABL14, KLS09, KKMS08, where at most a Ω̃(ϵ) fraction of labels can be flipped, our algorithm achieves a near-optimal label complexity of Õ(d1/ϵ) in time Õ(d^2/ϵ). Furthermore, we show that our active learning algorithm can be converted to an efficient passive learning algorithm that has near-optimal sample complexities with respect to ϵ and d.
READ FULL TEXT