On the reliability of published findings using the regression discontinuity design in political science

by   Drew Stommes, et al.

The regression discontinuity (RD) design offers identification of causal effects under weak assumptions, earning it the position as a standard method in modern political science research. But identification does not necessarily imply that the causal effects can be estimated accurately with limited data. In this paper, we highlight that estimation is particularly challenging with the RD design and investigate how these challenges manifest themselves in the empirical literature. We collect all RD-based findings published in top political science journals from 2009–2018. The findings exhibit pathological features; estimates tend to bunch just above the conventional level of statistical significance. A reanalysis of all studies with available data suggests that researcher's discretion is not a major driver of these pathological features, but researchers tend to use inappropriate methods for inference, rendering standard errors artificially small. A retrospective power analysis reveals that most of these studies were underpowered to detect all but large effects. The issues we uncover, combined with well-documented selection pressures in academic publishing, cause concern that many published findings using the RD design are exaggerated, if not entirely spurious.



There are no comments yet.


page 1

page 2

page 3

page 4


From controlled to undisciplined data: estimating causal effects in the era of data science using a potential outcome framework

This paper discusses the fundamental principles of causal inference - th...

Amnesty Policy and Elite Persistence in the Postbellum South: Evidence from a Regression Discontinuity Design

This paper investigates the impact of Reconstruction-era amnesty policy ...

Private Causal Inference using Propensity Scores

The use of propensity score methods to reduce selection bias when determ...

Analysis of regression discontinuity designs using censored data

In medical settings, treatment assignment may be determined by a clinica...

Clinically Relevant Mediation Analysis using Controlled Indirect Effect

Mediation analysis allows one to use observational data to estimate the ...

Dyadic Regression

Dyadic data, where outcomes reflecting pairwise interaction among sample...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.