
On the Risk of MinimumNorm Interpolants and Restricted Lower Isometry of Kernels
We study the risk of minimumnorm interpolants of data in a Reproducing ...
read it

Generalization error of minimum weighted norm and kernel interpolation
We study the generalization error of functions that interpolate prescrib...
read it

Approximation properties of certain operatorinduced norms on Hilbert spaces
We consider a class of operatorinduced norms, acting as finitedimensio...
read it

Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
For supervised and unsupervised learning, positive definite kernels allo...
read it

On the robustness of certain norms
We study a family of norms defined for functions on an interval. These n...
read it

Native Banach spaces for splines and variational inverse problems
We propose a systematic construction of native Banach spaces for general...
read it

Approximate is Good Enough: Probabilistic Variants of Dimensional and Margin Complexity
We present and study approximate notions of dimensional and margin compl...
read it
Minimum complexity interpolation in random features models
Despite their many appealing properties, kernel methods are heavily affected by the curse of dimensionality. For instance, in the case of inner product kernels in ℝ^d, the Reproducing Kernel Hilbert Space (RKHS) norm is often very large for functions that depend strongly on a small subset of directions (ridge functions). Correspondingly, such functions are difficult to learn using kernel methods. This observation has motivated the study of generalizations of kernel methods, whereby the RKHS norm – which is equivalent to a weighted ℓ_2 norm – is replaced by a weighted functional ℓ_p norm, which we refer to as ℱ_p norm. Unfortunately, tractability of these approaches is unclear. The kernel trick is not available and minimizing these norms requires to solve an infinitedimensional convex problem. We study random features approximations to these norms and show that, for p>1, the number of random features required to approximate the original learning problem is upper bounded by a polynomial in the sample size. Hence, learning with ℱ_p norms is tractable in these cases. We introduce a proof technique based on uniform concentration in the dual, which can be of broader interest in the study of overparametrized models.
READ FULL TEXT
Comments
There are no comments yet.