Phase Transition Unbiased Estimation in High Dimensional Settings

07/25/2019
by   Stéphane Guerrier, et al.
0

An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. For example, the maximum likelihood estimator has a bias that can result in a significant inferential loss. This problem is typically magnified in high-dimensional settings where the number of variables p is allowed to diverge with the sample size n. However, it is generally difficult to establish whether an estimator is unbiased and therefore its asymptotic order is a common approach used (in low-dimensional settings) to quantify the magnitude of the bias. As an alternative, we introduce a new and stronger property, possibly for high-dimensional settings, called phase transition unbiasedness. An estimator satisfying this property is unbiased for all n greater than a finite sample size n^∗. Moreover, we propose a phase transition unbiased estimator built upon the idea of matching an initial estimator computed on the sample and on simulated data. It is not required for this initial estimator to be consistent and thus it can be chosen for its computational efficiency and/or for other desirable properties such as robustness. This estimator can be computed using a suitable simulation based algorithm, namely the iterative bootstrap, which is shown to converge exponentially fast. In addition, we demonstrate the consistency and the limiting distribution of this estimator in high-dimensional settings. Finally, as an illustration, we use our approach to develop new estimators for the logistic regression model, with and without random effects, that also enjoy other properties such as robustness to data contamination and are also not affected by the problem of separability. In a simulation exercise, the theoretical results are confirmed in settings where the sample size is relatively small compared to the model dimension.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/19/2020

Asymptotically Optimal Bias Reduction for Parametric Models

An important challenge in statistical analysis concerns the control of t...
research
10/10/2018

On the Properties of Simulation-based Estimators in High Dimensions

Considering the increasing size of available data, the need for statisti...
research
03/08/2018

Aggregation using input-output trade-off

In this paper, we introduce a new learning strategy based on a seminal i...
research
10/26/2020

A General Approach for Simulation-based Bias Correction in High Dimensional Settings

An important challenge in statistical analysis lies in controlling the b...
research
05/11/2022

A zero-estimator approach for estimating the signal level in a high-dimensional model-free setting

We study a high-dimensional regression setting under the assumption of k...
research
02/18/2022

On Variance Estimation of Random Forests

Ensemble methods, such as random forests, are popular in applications du...
research
10/22/2020

Sharp Bias-variance Tradeoffs of Hard Parameter Sharing in High-dimensional Linear Regression

Hard parameter sharing for multi-task learning is widely used in empiric...

Please sign up or login with your details

Forgot password? Click here to reset