Hypothesis Tests That Are Robust to Choice of Matching Method

12/05/2018
by   Marco Morucci, et al.
0

A vast number of causal inference studies test hypotheses on treatment effects after treatment cases are matched with similar control cases. The quality of matched data is usually evaluated according to some metric, such as balance; however the same level of match quality can be achieved by different matches on the same data. Crucially, matches that achieve the same level of quality might lead to different results for hypothesis tests conducted on the matched data. Experimenters often specifically choose not to consider the uncertainty stemming from how the matches were constructed; this allows for easier computation and clearer testing, but it does not consider possible biases in the way the assignments were constructed. What we would really like to be able to report is that no matter which assignment we choose, as long as the match is sufficiently good, then the hypothesis test result still holds. In this paper, we provide methodology based on discrete optimization to create robust tests that explicitly account for this variation. For binary data, we give both fast algorithms to compute our tests and formulas for the null distributions of our test statistics under different conceptions of matching. For continuous data, we formulate a robust test statistic, and offer a linearization that permits faster computation. We apply our methods to real-world datasets and show that they can produce useful results in practical applied settings.

READ FULL TEXT
research
09/06/2020

Matching Bounds: How Choice of Matching Algorithm Impacts Treatment Effects Estimates and What to Do about It

Different matches on the same data may produce different treatment effec...
research
05/02/2023

Validation of massively-parallel adaptive testing using dynamic control matching

A/B testing is a widely-used paradigm within marketing optimization beca...
research
04/27/2022

On randomization inference after inexact Mahalanobis matching

In observational causal inference, matched-pairs studies are often analy...
research
07/19/2017

FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference

A classical problem in causal inference is that of matching, where treat...
research
12/22/2020

Algorithms for Solving Nonlinear Binary Optimization Problems in Robust Causal Inference

Identifying cause-effect relation among variables is a key step in the d...
research
12/29/2017

RedDwarfData: a simplified dataset of StarCraft matches

The game Starcraft is one of the most interesting arenas to test new mac...
research
06/18/2018

Collapsing-Fast-Large-Almost-Matching-Exactly: A Matching Method for Causal Inference

We aim to create the highest possible quality of treatment-control match...

Please sign up or login with your details

Forgot password? Click here to reset