Finite samples inference and critical dimension for stochastically linear models

01/17/2022
by   Vladimir Spokoiny, et al.
0

The aim of this note is to state a couple of general results about the properties of the penalized maximum likelihood estimators (pMLE) and of the posterior distribution for parametric models in a non-asymptotic setup and for possibly large or even infinite parameter dimension. We consider a special class of stochastically linear smooth (SLS) models satisfying two major conditions: the stochastic component of the log-likelihood is linear in the model parameter, while the expected log-likelihood is a smooth function. The main results simplify a lot if the expected log-likelihood is concave. For the pMLE, we establish a number of finite sample bounds about its concentration and large deviations as well as the Fisher and Wilks expansion. The later results extend the classical asymptotic Fisher and Wilks Theorems about the MLE to the non-asymptotic setup with large parameter dimension which can depend on the sample size. For the posterior distribution, our main result states a Gaussian approximation of the posterior which can be viewed as a finite sample analog of the prominent Bernstein–von Mises Theorem. In all bounds, the remainder is given explicitly and can be evaluated in terms of the effective sample size and effective parameter dimension. The results are dimension and coordinate free. In spite of generality, all the presented bounds are nearly sharp and the classical asymptotic results can be obtained as simple corollaries. An interesting case of logit regression with smooth or truncation priors is used to specify the results and to explain the main notions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/14/2019

Accuracy of Gaussian approximation in nonparametric Bernstein – von Mises Theorem

The prominent Bernstein – von Mises (BvM) result claims that the posteri...
research
09/03/2019

A Diffusion Process Perspective on Posterior Contraction Rates for Parameters

We show that diffusion processes can be exploited to study the posterior...
research
05/16/2020

Nonasymptotic Laplace approximation under model misspecification

We present non-asymptotic two-sided bounds to the log-marginal likelihoo...
research
05/14/2023

Nonlinear regression: finite sample guarantees

This paper offers a new approach for study the frequentist properties of...
research
09/13/2018

Deterministic Inequalities for Smooth M-estimators

Ever since the proof of asymptotic normality of maximum likelihood estim...
research
02/14/2023

Finite-sample bounds to the normal limit under group sequential sampling

In group sequential analysis, data is collected and analyzed in batches ...
research
06/11/2021

On an Asymptotic Distribution for the MLE

The paper presents a novel asymptotic distribution for a mle when the lo...

Please sign up or login with your details

Forgot password? Click here to reset