Extending the small-ball method

09/04/2017
by   Shahar Mendelson, et al.
0

The small-ball method was introduced as a way of obtaining a high probability, isomorphic lower bound on the quadratic empirical process, under weak assumptions on the indexing class. The key assumption was that class members satisfy a uniform small-ball estimate, that is, Pr(|f| ≥κf_L_2) ≥δ for given constants κ and δ. Here we extend the small-ball method and obtain a high probability, almost-isometric (rather than isomorphic) lower bound on the quadratic empirical process. The scope of the result is considerably wider than the small-ball method: there is no need for class members to satisfy a uniform small-ball condition, and moreover, motivated by the notion of tournament learning procedures, the result is stable under a `majority vote'. As applications, we study the performance of empirical risk minimization in learning problems involving bounded subsets of L_p that satisfy a Bernstein condition, and of the tournament procedure in problems involving bounded subsets of L_∞.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/22/2020

Approximate Covering with Lower and Upper Bounds via LP Rounding

In this paper, we study the lower- and upper-bounded covering (LUC) prob...
research
01/01/2014

Learning without Concentration

We obtain sharp bounds on the performance of Empirical Risk Minimization...
research
08/09/2023

Mean-Biased Processes for Balanced Allocations

We introduce a new class of balanced allocation processes which bias tow...
research
04/17/2019

Stable recovery and the coordinate small-ball behaviour of random vectors

Recovery procedures in various application in Data Science are based on ...
research
10/02/2018

Statistical learning with Lipschitz and convex loss functions

We obtain risk bounds for Empirical Risk Minimizers (ERM) and minmax Med...
research
04/19/2022

Compressed Empirical Measures (in finite dimensions)

We study approaches for compressing the empirical measure in the context...
research
01/04/2021

Benign overfitting without concentration

We obtain a sufficient condition for benign overfitting of linear regres...

Please sign up or login with your details

Forgot password? Click here to reset