The Bayesian Synthetic Control: Improved Counterfactual Estimation in the Social Sciences through Probabilistic Modeling
share
Social scientists often study how a policy reform impacted a single targeted country. Increasingly, this is done with the synthetic control method (SCM). SCM models the country's counterfactual (non-reform or untreated) trajectory as a weighted average of other countries' outcomes. The method struggles to quantify uncertainty; eg. it cannot produce confidence intervals. It is also suspect to overfit. We propose an alternative method, the Bayesian synthetic control (BSC), which lacks these flaws. Using MCMC sampling, we implement the method for two previously studied datasets. The proposed method outperforms SCM in a simple test of predictive accuracy and casts some doubt on significance of prior findings. The studied reforms are the German reunification of 1990 and the California tobacco legislation of 1988. BSC borrows its causal model, the linear latent factor model, from the SCM literature. Unlike SCM, BSC estimates the latent factors explicitly through a dimensionality reduction. All uncertainty is captured in the posterior distribution so that, unlike for SCM, credible intervals are easily derived. Further, BSC's reliability on the target panel dataset can be assessed through a posterior predictive check; SCM and its frequentist derivatives use up the required information while testing statistical significance.
READ FULL TEXT