On Agnostic PAC Learning using ℒ_2-polynomial Regression and Fourier-based Algorithms

02/11/2021
by   Mohsen Heidari, et al.
8

We develop a framework using Hilbert spaces as a proxy to analyze PAC learning problems with structural properties. We consider a joint Hilbert space incorporating the relation between the true label and the predictor under a joint distribution D. We demonstrate that agnostic PAC learning with 0-1 loss is equivalent to an optimization in the Hilbert space domain. With our model, we revisit the PAC learning problem using methods based on least-squares such as ℒ_2 polynomial regression and Linial's low-degree algorithm. We study learning with respect to several hypothesis classes such as half-spaces and polynomial-approximated classes (i.e., functions approximated by a fixed-degree polynomial). We prove that (under some distributional assumptions) such methods obtain generalization error up to 2opt with opt being the optimal error of the class. Hence, we show the tightest bound on generalization error when opt≤ 0.2.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/08/2023

Agnostic PAC Learning of k-juntas Using L2-Polynomial Regression

Many conventional learning algorithms rely on loss functions other than ...
research
10/24/2022

A PAC-Bayesian Generalization Bound for Equivariant Networks

Equivariant networks capture the inductive bias about the symmetry of th...
research
10/22/2020

The Polynomial Method is Universal for Distribution-Free Correlational SQ Learning

We consider the problem of distribution-free learning for Boolean functi...
research
11/28/2022

PAC Verification of Statistical Algorithms

Goldwasser et al. (2021) recently proposed the setting of PAC verificati...
research
03/03/2023

On the complexity of PAC learning in Hilbert spaces

We study the problem of binary classification from the point of view of ...
research
05/20/2021

Multi-group Agnostic PAC Learnability

An agnostic PAC learning algorithm finds a predictor that is competitive...
research
04/16/2022

PAC-Bayesian Based Adaptation for Regularized Learning

In this paper, we propose a PAC-Bayesian a posteriori parameter selectio...

Please sign up or login with your details

Forgot password? Click here to reset