Fast rates for empirical risk minimization with cadlag losses with bounded sectional variation norm

07/22/2019
by   Aurélien F. Bibaut, et al.
0

Empirical risk minimization over sieves of the class F of cadlag functions with bounded variation norm has a long history, starting with Total Variation Denoising (Rudin et al., 1992), and has been considered by several recent articles, in particular Fang et al. (2019) and van der Laan (2015). In this article, we show how a certain representation of cadlag functions with bounded sectional variation, also called Hardy-Krause variation, allows to bound the bracketing entropy of sieves of F and therefore derive fast rates of convergence in nonparametric function estimation. Specifically, for any sequence a_n that (slowly) diverges to ∞, we show that we can construct an estimator with rate of convergence O_P(2^d/3 n^-1/3 ( n)^d/3 a_n^2/3 ) over F, under some fairly general assumptions. Remarkably, the dimension only affects the rate in n through the logarithmic factor, making this method especially appropriate for high dimensional problems. In particular, we show that in the case of nonparametric regression over sieves of cadlag functions with bounded sectional variation norm, this upper bound on the rate of convergence holds for least-squares estimators, under the random design, sub-exponential errors setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/22/2019

Fast rates for empirical risk minimization over càdlàg functions with bounded sectional variation norm

Empirical risk minimization over classes functions that are bounded for ...
research
01/28/2021

Interpolating Classifiers Make Few Mistakes

This paper provides elementary analyses of the regret and generalization...
research
11/17/2019

Oracle inequalities for image denoising with total variation regularization

We derive oracle results for discrete image denoising with a total varia...
research
03/04/2019

Multivariate extensions of isotonic regression and total variation denoising via entire monotonicity and Hardy-Krause variation

We consider the problem of nonparametric regression when the covariate i...
research
05/26/2016

Total Variation Classes Beyond 1d: Minimax Rates, and the Limitations of Linear Smoothers

We consider the problem of estimating a function defined over n location...
research
12/08/2021

Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression

We study a localized notion of uniform convergence known as an "optimist...
research
06/03/2020

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

The least squares estimator (LSE) is shown to be suboptimal in squared e...

Please sign up or login with your details

Forgot password? Click here to reset