Log In Sign Up

Precise Unbiased Estimation in Randomized Experiments using Auxiliary Observational Data

by   Johann A. Gagnon-Bartsch, et al.

Randomized controlled trials (RCTs) are increasingly prevalent in education research, and are often regarded as a gold standard of causal inference. Two main virtues of randomized experiments are that they (1) do not suffer from confounding, thereby allowing for an unbiased estimate of an intervention's causal impact, and (2) allow for design-based inference, meaning that the physical act of randomization largely justifies the statistical assumptions made. However, RCT sample sizes are often small, leading to low precision; in many cases RCT estimates may be too imprecise to guide policy or inform science. Observational studies, by contrast, have strengths and weaknesses complementary to those of RCTs. Observational studies typically offer much larger sample sizes, but may suffer confounding. In many contexts, experimental and observational data exist side by side, allowing the possibility of integrating "big observational data" with "small but high-quality experimental data" to get the best of both. Such approaches hold particular promise in the field of education, where RCT sample sizes are often small due to cost constraints, but automatic collection of observational data, such as in computerized educational technology applications, or in state longitudinal data systems (SLDS) with administrative data on hundreds of thousand of students, has made rich, high-dimensional observational data widely available. We outline an approach that allows one to employ machine learning algorithms to learn from the observational data, and use the resulting models to improve precision in randomized experiments. Importantly, there is no requirement that the machine learning models are "correct" in any sense, and the final experimental results are guaranteed to be exactly unbiased. Thus, there is no danger of confounding biases in the observational data leaking into the experiment.


page 1

page 2

page 3

page 4


Removing Hidden Confounding by Experimental Grounding

Observational data is increasingly used as a means for making individual...

Designing Experiments Toward Shrinkage Estimation

We consider how increasingly available observational data can be used to...

An introduction to causal reasoning in health analytics

A data science task can be deemed as making sense of the data and/or tes...

Adversarial De-confounding in Individualised Treatment Effects Estimation

Observational studies have recently received significant attention from ...

Combining Observational and Experimental Data Using First-stage Covariates

Randomized controlled trials generate experimental variation that can cr...

Designing Experiments Informed by Observational Studies

The increasing availability of passively observed data has yielded a gro...