Benign-Overfitting in Conditional Average Treatment Effect Prediction with Linear Regression

by   Masahiro Kato, et al.

We study the benign overfitting theory in the prediction of the conditional average treatment effect (CATE), with linear regression models. As the development of machine learning for causal inference, a wide range of large-scale models for causality are gaining attention. One problem is that suspicions have been raised that the large-scale models are prone to overfitting to observations with sample selection, hence the large models may not be suitable for causal prediction. In this study, to resolve the suspicious, we investigate on the validity of causal inference methods for overparameterized models, by applying the recent theory of benign overfitting (Bartlett et al., 2020). Specifically, we consider samples whose distribution switches depending on an assignment rule, and study the prediction of CATE with linear models whose dimension diverges to infinity. We focus on two methods: the T-learner, which based on a difference between separately constructed estimators with each treatment group, and the inverse probability weight (IPW)-learner, which solves another regression problem approximated by a propensity score. In both methods, the estimator consists of interpolators that fit the samples perfectly. As a result, we show that the T-learner fails to achieve the consistency except the random assignment, while the IPW-learner converges the risk to zero if the propensity score is known. This difference stems from that the T-learner is unable to preserve eigenspaces of the covariances, which is necessary for benign overfitting in the overparameterized setting. Our result provides new insights into the usage of causal inference methods in the overparameterizated setting, in particular, doubly robust estimators.


page 1

page 2

page 3

page 4


On the implied weights of linear regression for causal inference

In this paper, we derive and analyze the implied weights of linear regre...

Towards R-learner of conditional average treatment effects with a continuous treatment: T-identification, estimation, and inference

The R-learner has been popular in causal inference as a flexible and eff...

Machine learning in policy evaluation: new tools for causal inference

While machine learning (ML) methods have received a lot of attention in ...

SDRcausal: an R package for causal inference based on sufficient dimension reduction

SDRcausal is a package that implements sufficient dimension reduction me...

Doubly Robust Criterion for Causal Inference

The semiparametric estimation approach, which includes inverse-probabili...

Inverse Probability Weighting: the Missing Link between Survey Sampling and Evidence Estimation

We consider the class of inverse probability weight (IPW) estimators, in...

Unbiased and Efficient Estimation of Causal Treatment Effects in Cross-over Trials

We introduce causal inference reasoning to cross-over trials, with a foc...