Treatment Effect Risk: Bounds and Inference

by   Nathan Kallus, et al.

Since the average treatment effect (ATE) measures the change in social welfare, even if positive, there is a risk of negative effect on, say, some 10 of the population. Assessing such risk is difficult, however, because any one individual treatment effect (ITE) is never observed so the 10 cannot be identified, while distributional treatment effects only compare the first deciles within each treatment group, which does not correspond to any 10 important risk measure, formalized as the conditional value at risk (CVaR) of the ITE distribution. We leverage the availability of pre-treatment covariates and characterize the tightest-possible upper and lower bounds on ITE-CVaR given by the covariate-conditional average treatment effect (CATE) function. Some bounds can also be interpreted as summarizing a complex CATE function into a single metric and are of interest independently of being a bound. We then proceed to study how to estimate these bounds efficiently from data and construct confidence intervals. This is challenging even in randomized experiments as it requires understanding the distribution of the unknown CATE function, which can be very complex if we use rich covariates so as to best control for heterogeneity. We develop a debiasing method that overcomes this and prove it enjoys favorable statistical properties even when CATE and other nuisances are estimated by black-box machine learning or even inconsistently. Studying a hypothetical change to French job-search counseling services, our bounds and inference demonstrate a small social benefit entails a negative impact on a substantial subpopulation.



page 13


Bounds on the conditional and average treatment effect in the presence of unobserved confounders

The causal effect of an intervention can not be consistently estimated w...

Sharp Bounds for the Marginal Treatment Effect with Sample Selection

I analyze treatment effects in situations when agents endogenously selec...

Non-asymptotic confidence bands on the probability an individual benefits from treatment (PIBT)

The premise of this work, in a vein similar to predictive inference with...

Robust and Agnostic Learning of Conditional Distributional Treatment Effects

The conditional average treatment effect (CATE) is the best point predic...

Evaluating Treatment Prioritization Rules via Rank-Weighted Average Treatment Effects

There are a number of available methods that can be used for choosing wh...

Conditional Distributional Treatment Effect with Kernel Conditional Mean Embeddings and U-Statistic Regression

We propose to analyse the conditional distributional treatment effect (C...

Conformal Inference of Counterfactuals and Individual Treatment Effects

Evaluating treatment effect heterogeneity widely informs treatment decis...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.