On the Risk of Minimum-Norm Interpolants and Restricted Lower Isometry of Kernels

08/27/2019

∙

by   Tengyuan Liang, et al.

∙

6

∙

share

We study the risk of minimum-norm interpolants of data in a Reproducing Kernel Hilbert Space where kernel is defined as a function of the inner product. Our upper bounds on the risk are of a multiple-descent shape for the various scalings of d = n^α, α∈(0,1), for the input dimension d and sample size n. At the heart of our analysis is a study of spectral properties of the random kernel matrix restricted to a filtration of eigen-spaces of the population covariance operator. Since gradient flow on appropriately initialized wide neural networks converges to a minimum-norm interpolant, the analysis also yields estimation guarantees for these models.