On the robustness of minimum-norm interpolators

12/01/2020
by   Geoffrey Chinot, et al.
1

This article develops a general theory for minimum-norm interpolated estimators in linear models in the presence of additive, potentially adversarial, errors. In particular, no conditions on the errors are imposed. A quantitative bound for the prediction error is given, relating it to the Rademacher complexity of the covariates, the norm of the minimum norm interpolator of the errors and the shape of the subdifferential around the true parameter. The general theory is illustrated with several examples: the sparse linear model with minimum ℓ_1-norm or group Lasso penalty interpolation, the low rank trace regression model with nuclear norm minimization, and minimum Euclidean norm interpolation in the linear model. In case of sparsity or low-rank inducing norms, minimum norm interpolation yields a prediction error of the order of the average noise level, provided that the overparameterization is at least a logarithmic factor larger than the number of samples. Lower bounds that show near optimality of the results complement the analysis.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/10/2021

Tight bounds for minimum l1-norm interpolation of noisy data

We provide matching upper and lower bounds of order σ^2/log(d/n) for the...
12/21/2018

Low-rank Approximation of Linear Maps

This work provides closed-form solutions and minimal achievable errors f...
08/07/2020

Generalization error of minimum weighted norm and kernel interpolation

We study the generalization error of functions that interpolate prescrib...
09/09/2011

Trace Lasso: a trace norm regularization for correlated designs

Using the ℓ_1-norm to regularize the estimation of the parameter vector ...
08/28/2010

On Euclidean Norm Approximations

Euclidean norm calculations arise frequently in scientific and engineeri...
10/17/2018

Efficient Proximal Mapping Computation for Unitarily Invariant Low-Rank Inducing Norms

Low-rank inducing unitarily invariant norms have been introduced to conv...
06/10/2020

On Uniform Convergence and Low-Norm Interpolation Learning

We consider an underdetermined noisy linear regression model where the m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.