# All of Linear Regression

Least squares linear regression is one of the oldest and widely used data analysis tools. Although the theoretical analysis of the ordinary least squares (OLS) estimator is as old, several fundamental questions are yet to be answered. Suppose regression observations (X_1,Y_1),...,(X_n,Y_n)∈R^d×R (not necessarily independent) are available. Some of the questions we deal with are as follows: under what conditions, does the OLS estimator converge and what is the limit? What happens if the dimension is allowed to grow with n? What happens if the observations are dependent with dependence possibly strengthening with n? How to do statistical inference under these kinds of misspecification? What happens to the OLS estimator under variable selection? How to do inference under misspecification and variable selection? We answer all the questions raised above with one simple deterministic inequality which holds for any set of observations and any sample size. This implies that all our results are a finite sample (non-asymptotic) in nature. In the end, one only needs to bound certain random quantities under specific settings of interest to get concrete rates and we derive these bounds for the case of independent observations. In particular, the problem of inference after variable selection is studied, for the first time, when d, the number of covariates increases (almost exponentially) with sample size n. We provide comments on the “right” statistic to consider for inference under variable selection and efficient computation of quantiles.

• 8 publications
• 6 publications
• 8 publications
• 4 publications
research
07/03/2023

### Inference for Projection Parameters in Linear Regression: beyond d = o(n^1/2)

We consider the problem of inference for projection parameters in linear...
research
02/15/2018

### A Model Free Perspective for Linear Regression: Uniform-in-model Bounds for Post Selection Inference

For the last two decades, high-dimensional data and methods have prolife...
research
10/05/2020

### A Power Analysis of the Conditional Randomization Test and Knockoffs

In many scientific problems, researchers try to relate a response variab...
research
08/22/2023

### Nonparametric Assessment of Variable Selection and Ranking Algorithms

Selecting from or ranking a set of candidates variables in terms of thei...
research
09/03/2022

### Forbidden Knowledge and Specialized Training: A Versatile Solution for the Two Main Sources of Overfitting in Linear Regression

Overfitting in linear regression is broken down into two main causes. Fi...
research
09/07/2015

### Poisson Subsampling Algorithms for Large Sample Linear Regression in Massive Data

Large sample size brings the computation bottleneck for modern data anal...
research
06/05/2019

### A Model-free Approach to Linear Least Squares Regression with Exact Probabilities and Applications to Covariate Selection

The classical model for linear regression is Y= xβ +σε with i.i.d. stan...