Oracle Inequalities for Convex Loss Functions with Non-Linear Targets

12/12/2013
by   Mehmet Caner, et al.
0

This paper consider penalized empirical loss minimization of convex loss functions with unknown non-linear target functions. Using the elastic net penalty we establish a finite sample oracle inequality which bounds the loss of our estimator from above with high probability. If the unknown target is linear this inequality also provides an upper bound of the estimation error of the estimated parameter vector. These are new results and they generalize the econometrics and statistics literature. Next, we use the non-asymptotic results to show that the excess loss of our estimator is asymptotically of the same order as that of the oracle. If the target is linear we give sufficient conditions for consistency of the estimated parameter vector. Next, we briefly discuss how a thresholded version of our estimator can be used to perform consistent variable selection. We give two examples of loss functions covered by our framework and show how penalized nonparametric series estimation is contained as a special case and provide a finite sample upper bound on the mean square error of the elastic net series estimator.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/09/2017

Elastic-net regularized High-dimensional Negative Binomial Regression: Consistency and Weak Signals Detection

We study sparse high-dimensional negative binomial regression problem fo...
research
08/30/2019

Consistency and Finite Sample Behavior of Binary Class Probability Estimation

In this work we investigate to which extent one can recover class probab...
research
02/21/2015

Learning with Square Loss: Localization through Offset Rademacher Complexity

We consider regression with square loss and general classes of functions...
research
08/16/2018

Sparse Multivariate ARCH Models: Finite Sample Properties

We provide finite sample properties of sparse multivariate ARCH processe...
research
04/29/2021

Generalized Linear Models with Structured Sparsity Estimators

In this paper, we introduce structured sparsity estimators in Generalize...
research
10/07/2018

Error bounds for sparse classifiers in high-dimensions

We prove an L2 recovery bound for a family of sparse estimators defined ...
research
12/03/2019

Bayesian Model Selection for Change Point Detection and Clustering

We address the new problem of estimating a piece-wise constant signal wi...

Please sign up or login with your details

Forgot password? Click here to reset