DeepAI AI Chat
Log In Sign Up

PAC-Bayesian Based Adaptation for Regularized Learning

by   Prem Talwai, et al.

In this paper, we propose a PAC-Bayesian a posteriori parameter selection scheme for adaptive regularized regression in Hilbert scales under general, unknown source conditions. We demonstrate that our approach is adaptive to misspecification, and achieves the optimal learning rate under subgaussian noise. Unlike existing parameter selection schemes, the computational complexity of our approach is independent of sample size. We derive minimax adaptive rates for a new, broad class of Tikhonov-regularized learning problems under general, misspecified source conditions, that notably do not require any conventional a priori assumptions on kernel eigendecay. Using the theory of interpolation, we demonstrate that the spectrum of the Mercer operator can be inferred in the presence of "tight" L^∞ embeddings of suitable Hilbert scales. Finally, we prove, that under a Δ_2 condition on the smoothness index functions, our PAC-Bayesian scheme can indeed achieve minimax rates. We discuss applications of our approach to statistical inverse problems and oracle-efficient contextual bandit algorithms.


page 1

page 2

page 3

page 4


Inverse learning in Hilbert scales

We study the linear ill-posed inverse problem with noisy data in the sta...

Tikhonov regularization with oversmoothing penalty for nonlinear statistical inverse problems

In this paper, we consider the nonlinear ill-posed inverse problem with ...

Statistical Inverse Problems in Hilbert Scales

In this paper, we study the Tikhonov regularization scheme in Hilbert sc...

Adaptive minimax optimality in statistical inverse problems via SOLIT – Sharp Optimal Lepskii-Inspired Tuning

We consider statistical linear inverse problems in separable Hilbert spa...

Analysis of regularized Nyström subsampling for regression functions of low smoothness

This paper studies a Nyström type subsampling approach to large kernel l...

Optimal rates for the regularized learning algorithms under general source condition

We consider the learning algorithms under general source condition with ...

On Agnostic PAC Learning using ℒ_2-polynomial Regression and Fourier-based Algorithms

We develop a framework using Hilbert spaces as a proxy to analyze PAC le...