Uniform Deviation Bounds for Unbounded Loss Functions like k-Means

02/27/2017
by   Olivier Bachem, et al.
0

Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are *unbounded*. In our main application, this allows us to obtain bounds for k-Means clustering under weak assumptions on the underlying distribution. If the fourth moment is bounded, we prove a rate of O(m^-1/2) compared to the previously known O(m^-1/4) rate. Furthermore, we show that the rate also depends on the kurtosis - the normalized fourth moment which measures the "tailedness" of a distribution. We further provide improved rates under progressively stronger assumptions, namely, bounded higher moments, subgaussianity and bounded support.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/14/2020

Risk Bounds for Robust Deep Learning

It has been observed that certain loss functions can render deep-learnin...
research
09/07/2016

Chaining Bounds for Empirical Risk Minimization

This paper extends the standard chaining technique to prove excess risk ...
research
08/09/2023

Bounded Distributions place Limits on Skewness and Larger Moments

Distributions of strictly positive numbers are common and can be charact...
research
03/12/2020

Benign overfitting in the large deviation regime

We investigate the benign overfitting phenomenon in the large deviation ...
research
05/01/2016

Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayes

We present new excess risk bounds for general unbounded loss functions i...
research
01/06/2022

Robust Linear Predictions: Analyses of Uniform Concentration, Fast Rates and Model Misspecification

The problem of linear predictions has been extensively studied for the p...
research
09/26/2013

Bennett-type Generalization Bounds: Large-deviation Case and Faster Rate of Convergence

In this paper, we present the Bennett-type generalization bounds of the ...

Please sign up or login with your details

Forgot password? Click here to reset