Methodological considerations for estimating policy effects in the context of co-occurring policies

06/08/2021
by   Beth Ann Griffin, et al.
0

Objective. Understanding how best to estimate state-level policy effects is important, and several unanswered questions remain, particularly about optimal methods for disentangling the effects of concurrently implemented policies. In this paper, we examined the impact of co-occurring policies on the performance of commonly used models in state policy evaluations. Data Sources. Outcome of interest (annual state-specific opioid mortality rate per 100,000) was obtained from 1999-2016 National Vital Statistics System (NVSS) Multiple Cause of Death mortality files. Study Design. We utilized Monte Carlo simulations to assess the effect of concurrent policy enactment on the evaluation of state-level policies. Simulation conditions varied effect sizes of the co-occurring policies as well as the length of time between enactment dates of the co-occurring policies, among other factors. Data Collection. Longitudinal annual state-level data over 18 years from 50 states. Principal Findings. Our results demonstrated high relative bias (>85 arise when confounding co-occurring policies are omitted from the analytic model and the co-occuring policies are enacted in rapid succession. Moreover, our findings indicated that controlling for all co-occurring policies will effectively mitigate the threat of confounding bias; however, effect estimates may be relatively imprecise, with larger variance estimates when co-occurring policies were enacted in near succession of each other. We also found that the required length of time between co-occurring policies necessary to obtain robust policy estimates varied across model specifications, being generally shorter for autoregressive (AR) models compared to difference-in-differences (DID) models.

READ FULL TEXT

page 18

page 20

page 21

research
07/04/2023

Identifying Optimal Methods for Addressing Confounding Bias When Estimating the Effects of State-Level Policies

Background: Policy evaluation studies that assess how state-level polici...
research
06/07/2018

Understanding State level Variations in U.S. Infant Mortality: 2000 to 2015

Objective: To exploit state variations in infant mortality, identify dia...
research
11/11/2020

A trial emulation approach for policy evaluations with group-level longitudinal data

To limit the spread of the novel coronavirus, governments across the wor...
research
09/07/2021

Estimating effects within nonlinear autoregressive models: a case study on the impact of child access prevention laws on firearm mortality

Autoregressive models are widely used for the analysis of time-series da...
research
06/23/2022

Inference on the Best Policies with Many Covariates

Understanding the impact of the most effective policies or treatments on...
research
10/21/2018

Predicting the outcomes of policy diffusion from U.S. states to federal law

In the United States, national policies often begin as state laws, which...

Please sign up or login with your details

Forgot password? Click here to reset