Synth-Validation: Selecting the Best Causal Inference Method for a Given Dataset

10/31/2017
by   Alejandro Schuler, et al.
0

Many decisions in healthcare, business, and other policy domains are made without the support of rigorous evidence due to the cost and complexity of performing randomized experiments. Using observational data to answer causal questions is risky: subjects who receive different treatments also differ in other ways that affect outcomes. Many causal inference methods have been developed to mitigate these biases. However, there is no way to know which method might produce the best estimate of a treatment effect in a given study. In analogy to cross-validation, which estimates the prediction error of predictive models applied to a given dataset, we propose synth-validation, a procedure that estimates the estimation error of causal inference methods applied to a given dataset. In synth-validation, we use the observed data to estimate generative distributions with known treatment effects. We apply each causal inference method to datasets sampled from these distributions and compare the effect estimates with the known effects to estimate error. Using simulations, we show that using synth-validation to select a causal inference method for each study lowers the expected estimation error relative to consistently using any single method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/14/2018

Benchmarking Framework for Performance-Evaluation of Causal Inference Analysis

Causal inference analysis is the estimation of the effects of actions on...
research
02/01/2023

How to select predictive models for causal inference?

Predictive models – as with machine learning – can underpin causal infer...
research
04/05/2015

Recursive Partitioning for Heterogeneous Causal Effects

In this paper we study the problems of estimating heterogeneity in causa...
research
11/11/2020

Teaching deep learning causal effects improves predictive performance

Causal inference is a powerful statistical methodology for explanatory a...
research
12/22/2019

Using Small Domain Estimation to obtain better retrospective Age Period Cohort insights

Recent changes in housing costs relative to income are likely to affect ...
research
12/24/2017

Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms

We investigate the problem of estimating the causal effect of a treatmen...
research
10/04/2021

When can relative risks provide causal estimates?

It is emphasised that for epidemiological studies where disease incidenc...

Please sign up or login with your details

Forgot password? Click here to reset