Deterministic Inequalities for Smooth M-estimators

09/13/2018
by   Arun Kumar Kuchibhotla, et al.
0

Ever since the proof of asymptotic normality of maximum likelihood estimator by Cramer (1946), it has been understood that a basic technique of the Taylor series expansion suffices for asymptotics of M-estimators with smooth/differentiable loss function. Although the Taylor series expansion is a purely deterministic tool, the realization that the asymptotic normality results can also be made deterministic (and so finite sample) received far less attention. With the advent of big data and high-dimensional statistics, the need for finite sample results has increased. In this paper, we use the (well-known) Banach fixed point theorem to derive various deterministic inequalities that lead to the classical results when studied under randomness. In addition, we provide applications of these deterministic inequalities for crossvalidation/subsampling, marginal screening and uniform-in-submodel results that are very useful for post-selection inference and in the study of post-regularization estimators. Our results apply to many classical estimators, in particular, generalized linear models, non-linear regression and cox proportional hazards model. Extensions to non-smooth and constrained problems are also discussed.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/15/2018

A Model Free Perspective for Linear Regression: Uniform-in-model Bounds for Post Selection Inference

For the last two decades, high-dimensional data and methods have prolife...
research
04/09/2012

Non-asymptotic Oracle Inequalities for the High-Dimensional Cox Regression via Lasso

We consider the finite sample properties of the regularized high-dimensi...
research
01/17/2022

Finite samples inference and critical dimension for stochastically linear models

The aim of this note is to state a couple of general results about the p...
research
09/29/2017

Robust Estimation in High Dimensional Generalized Linear Models

Generalized Linear Models are routinely used in data analysis. The class...
research
10/24/2021

Robust Strongly Convergent M-Estimators Under Non-IID Assumption

M-estimators for Generalized Linear Models are considered under minimal ...
research
09/02/2019

Asymptotic linear expansion of regularized M-estimators

Parametric high-dimensional regression analysis requires the usage of re...
research
06/20/2021

Some smooth sequential empirical copula processes and their multiplier bootstraps under strong mixing

A broad class of smooth empirical copulas that contains the empirical be...

Please sign up or login with your details

Forgot password? Click here to reset