Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization

06/16/2021
by   Gaspard Beugnot, et al.
0

The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels. For least squares, it allows to derive various regularization schemes that yield faster convergence rates of the excess risk than with Tikhonov regularization. This is typically achieved by leveraging classical assumptions called source and capacity conditions, which characterize the difficulty of the learning task. In order to understand estimators derived from other loss functions, Marteau-Ferey et al. have extended the theory of Tikhonov regularization to generalized self concordant loss functions (GSC), which contain, e.g., the logistic loss. In this paper, we go a step further and show that fast and optimal rates can be achieved for GSC by using the iterated Tikhonov regularization scheme, which is intrinsically related to the proximal point method in optimization, and overcomes the limitation of the classical Tikhonov regularization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2019

Beyond Least-Squares: Fast Rates for Regularized Empirical Risk Minimization through Self-Concordance

We consider learning methods based on the regularization of a convex emp...
research
03/31/2015

Iterative Regularization for Learning with Convex Loss Functions

We consider the problem of supervised learning with convex loss function...
research
05/01/2016

Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes

We present new excess risk bounds for general unbounded loss functions i...
research
07/18/2019

On the relation between Loss Functions and T-Norms

Deep learning has been shown to achieve impressive results in several do...
research
10/20/2019

Sparse (group) learning with Lipschitz loss functions: a unified analysis

We study a family of sparse estimators defined as minimizers of some emp...
research
12/19/2013

Learning rates of l^q coefficient regularization learning with Gaussian kernel

Regularization is a well recognized powerful strategy to improve the per...
research
08/17/2022

Characterizing M-estimators

We characterize the full classes of M-estimators for semiparametric mode...

Please sign up or login with your details

Forgot password? Click here to reset