Optimal Rates for Spectral-regularized Algorithms with Least-Squares Regression over Hilbert Spaces

01/20/2018
by   Junhong Lin, et al.
0

In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space. We investigate a class of spectral-regularized algorithms, including ridge regression, principal component analysis, and gradient methods. We prove optimal, high-probability convergence results in terms of variants of norms for the studied algorithms, considering a capacity assumption on the hypothesis space and a general source condition on the target function. Consequently, we obtain almost sure convergence results with optimal rates. Our results improve and generalize previous results, filling a theoretical gap for the non-attainable cases.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/12/2018

Optimal Rates of Sketched-regularized Algorithms for Least-Squares Regression over Hilbert Spaces

We investigate regularized algorithms combining with projection for leas...
research
01/22/2018

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral-Regularization Algorithms

We study generalization properties of distributed algorithms in the sett...
research
11/05/2018

Kernel Conjugate Gradient Methods with Random Projections

We propose and study kernel conjugate gradient methods (KCGM) with rando...
research
08/28/2022

Statistical Inverse Problems in Hilbert Scales

In this paper, we study the Tikhonov regularization scheme in Hilbert sc...
research
05/25/2019

Kernel Truncated Randomized Ridge Regression: Optimal Rates and Low Noise Acceleration

In this paper, we consider the nonparametric least square regression in ...
research
08/16/2023

Reproducing Kernel Hilbert Space Pruning for Sparse Hyperspectral Abundance Prediction

Hyperspectral measurements from long range sensors can give a detailed p...
research
06/01/2020

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

In this paper, we study the asymptotical properties of least squares reg...

Please sign up or login with your details

Forgot password? Click here to reset