Post-Selection Inference via Algorithmic Stability

11/18/2020
by   Tijana Zrnic, et al.
0

Modern approaches to data analysis make extensive use of data-driven model selection. The resulting dependencies between the selected model and data used for inference invalidate statistical guarantees derived from classical theories. The framework of post-selection inference (PoSI) has formalized this problem and proposed corrections which ensure valid inferences. Yet, obtaining general principles that enable computationally-efficient, powerful PoSI methodology with formal guarantees remains a challenge. With this goal in mind, we revisit the PoSI problem through the lens of algorithmic stability. Under an appropriate formulation of stability—one that captures closure under post-processing and compositionality properties—we show that stability parameters of a selection method alone suffice to provide non-trivial corrections to classical z-test and t-test intervals. Then, for several popular model selection methods, including the LASSO, we show how stability can be achieved through simple, computationally efficient randomization schemes. Our algorithms offer provable unconditional simultaneous coverage and are computationally efficient; in particular, they do not rely on MCMC sampling. Importantly, our proposal explicitly relates the magnitude of randomization to the resulting confidence interval width, allowing the analyst to tune interval width to the loss in utility due to randomizing selection.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2017

Two sources of poor coverage of confidence intervals after model selection

We compare the following two sources of poor coverage of post-model-sele...
research
05/21/2023

A parametric distribution for exact post-selection inference with data carving

Post-selection inference (PoSI) is a statistical technique for obtaining...
research
12/06/2017

On overfitting and post-selection uncertainty assessments

In a regression context, when the relevant subset of explanatory variabl...
research
06/24/2023

Post-Selection Inference for the Cox Model with Interval-Censored Data

We develop a post-selection inference method for the Cox proportional ha...
research
12/10/2018

Post-Selection Inference for Changepoint Detection Algorithms with Application to Copy Number Variation Data

Changepoint detection methods are used in many areas of science and engi...
research
12/29/2021

Exact Post-selection Inference For Tracking S P500

The problem that is solved in this paper is known as index tracking. The...
research
03/05/2021

Forward Stability and Model Path Selection

Most scientific publications follow the familiar recipe of (i) obtain da...

Please sign up or login with your details

Forgot password? Click here to reset