Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability

06/16/2022
by   Jonathan Crabbé, et al.
1

Estimating personalized effects of treatments is a complex, yet pervasive problem. To tackle it, recent developments in the machine learning (ML) literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools: due to their flexibility, modularity and ability to learn constrained representations, neural networks in particular have become central to this literature. Unfortunately, the assets of such black boxes come at a cost: models typically involve countless nontrivial operations, making it difficult to understand what they have learned. Yet, understanding these models can be crucial – in a medical context, for example, discovered knowledge on treatment effect heterogeneity could inform treatment prescription in clinical practice. In this work, we therefore use post-hoc feature importance methods to identify features that influence the model's predictions. This allows us to evaluate treatment effect estimators along a new and important dimension that has been overlooked in previous work: We construct a benchmarking environment to empirically investigate the ability of personalized treatment effect models to identify predictive covariates – covariates that determine differential responses to treatment. Our benchmarking environment then enables us to provide new insight into the strengths and weaknesses of different types of treatment effects models as we modulate different challenges specific to treatment effect estimation – e.g. the ratio of prognostic to predictive information, the possible nonlinearity of potential outcomes and the presence and type of confounding.

READ FULL TEXT
research
10/25/2018

Heterogeneous Treatment Effect Estimation through Deep Learning

Estimating heterogeneous treatment effect is an important task in causal...
research
03/07/2021

DINA: Estimating Heterogenous Treatment Effects in Exponential Family and Cox Model

We propose to use the difference in natural parameters (DINA) to quantif...
research
10/15/2019

Learning Sample-Specific Models with Low-Rank Personalized Regression

Modern applications of machine learning (ML) deal with increasingly hete...
research
07/14/2020

A unified survey on treatment effect heterogeneity modeling and uplift modeling

A central question in many fields of scientific research is to determine...
research
04/09/2021

Assessing the Impact of COVID-19 on Trade: a Machine Learning Counterfactual Analysis

By interpreting exporters' dynamics as a complex learning process, this ...
research
11/05/2021

Distilling Heterogeneity: From Explanations of Heterogeneous Treatment Effect Models to Interpretable Policies

Internet companies are increasingly using machine learning models to cre...

Please sign up or login with your details

Forgot password? Click here to reset