Approximate is Good Enough: Probabilistic Variants of Dimensional and Margin Complexity

by   Pritish Kamath, et al.

We present and study approximate notions of dimensional and margin complexity, which correspond to the minimal dimension or norm of an embedding required to approximate, rather then exactly represent, a given hypothesis class. We show that such notions are not only sufficient for learning using linear predictors or a kernel, but unlike the exact variants, are also necessary. Thus they are better suited for discussing limitations of linear or kernel methods.


page 1

page 2

page 3

page 4


Optimality of 1-norm regularization among weighted 1-norms for sparse recovery: a case study on how to find optimal regularizations

The 1-norm was proven to be a good convex regularizer for the recovery o...

On the Rademacher Complexity of Linear Hypothesis Sets

Linear predictors form a rich class of hypotheses used in a variety of l...

Kernel-based interpolation at approximate Fekete points

We construct approximate Fekete point sets for kernel-based interpolatio...

Exponential tractability of L_2-approximation with function values

We study the complexity of high-dimensional approximation in the L_2-nor...

Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a Margin

We study the problem of properly learning large margin halfspaces in th...

Polynomial magic! Hermite polynomials for private data generation

Kernel mean embedding is a useful tool to compare probability measures. ...

Approximate Vertex Enumeration

The problem to compute a V-polytope which is close to a given H-polytope...