On the robustness of minimum-norm interpolators

by   Geoffrey Chinot, et al.

This article develops a general theory for minimum-norm interpolated estimators in linear models in the presence of additive, potentially adversarial, errors. In particular, no conditions on the errors are imposed. A quantitative bound for the prediction error is given, relating it to the Rademacher complexity of the covariates, the norm of the minimum norm interpolator of the errors and the shape of the subdifferential around the true parameter. The general theory is illustrated with several examples: the sparse linear model with minimum ℓ_1-norm or group Lasso penalty interpolation, the low rank trace regression model with nuclear norm minimization, and minimum Euclidean norm interpolation in the linear model. In case of sparsity or low-rank inducing norms, minimum norm interpolation yields a prediction error of the order of the average noise level, provided that the overparameterization is at least a logarithmic factor larger than the number of samples. Lower bounds that show near optimality of the results complement the analysis.



There are no comments yet.


page 1

page 2

page 3

page 4


Tight bounds for minimum l1-norm interpolation of noisy data

We provide matching upper and lower bounds of order σ^2/log(d/n) for the...

Low-rank Approximation of Linear Maps

This work provides closed-form solutions and minimal achievable errors f...

Generalization error of minimum weighted norm and kernel interpolation

We study the generalization error of functions that interpolate prescrib...

Trace Lasso: a trace norm regularization for correlated designs

Using the ℓ_1-norm to regularize the estimation of the parameter vector ...

On Euclidean Norm Approximations

Euclidean norm calculations arise frequently in scientific and engineeri...

Efficient Proximal Mapping Computation for Unitarily Invariant Low-Rank Inducing Norms

Low-rank inducing unitarily invariant norms have been introduced to conv...

On Uniform Convergence and Low-Norm Interpolation Learning

We consider an underdetermined noisy linear regression model where the m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.