How Much Should We Trust Instrumental Variable Estimates in Political Science? Practical Advice Based on Over 60 Replicated Studies

03/20/2023
by   Apoorva Lal, et al.
0

Instrumental variable (IV) strategies are widely used in political science to establish causal relationships. However, the identifying assumptions required by an IV design are demanding, and it remains challenging for researchers to assess their validity. In this paper, we replicate 67 papers published in three top journals in political science during 2010-2022 and identify several troubling patterns. First, researchers often overestimate the strength of their IVs due to non-i.i.d. errors, such as a clustering structure. Second, the most commonly used t-test for the two-stage-least-squares (2SLS) estimates often severely underestimates uncertainty. Using more robust inferential methods, we find that around 19-30 Third, in the majority of the replicated studies, the 2SLS estimates are much larger than the ordinary-least-squares estimates, and their ratio is negatively correlated with the strength of the IVs in studies where the IVs are not experimentally generated, suggesting potential violations of unconfoundedness or the exclusion restriction. To help researchers avoid these pitfalls, we provide a checklist for better practice.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2021

On the reliability of published findings using the regression discontinuity design in political science

The regression discontinuity (RD) design offers identification of causal...
research
10/20/2021

One Instrument to Rule Them All: The Bias and Coverage of Just-ID IV

Two-stage least squares estimates in heavily over-identified instrumenta...
research
12/04/2018

Necessary and Probably Sufficient Test for Finding Valid Instrumental Variables

Can instrumental variables be found from data? While instrumental variab...
research
04/11/2020

Ivy: Instrumental Variable Synthesis for Causal Inference

A popular way to estimate the causal effect of a variable x on y from ob...
research
06/27/2019

Interpretable Almost-Matching-Exactly With Instrumental Variables

Uncertainty in the estimation of the causal effect in observational stud...
research
03/26/2021

Amnesty Policy and Elite Persistence in the Postbellum South: Evidence from a Regression Discontinuity Design

This paper investigates the impact of Reconstruction-era amnesty policy ...
research
01/21/2021

A practice-oriented overview of call center workforce planning

We give an overview of the practice and science of call center workforce...

Please sign up or login with your details

Forgot password? Click here to reset