Log In Sign Up

Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

by   Zihao Yang, et al.

Classical randomized experiments, equipped with randomization-based inference, provide assumption-free inference for treatment effects. They have been the gold standard for drawing causal inference and provide excellent internal validity. However, they have also been criticized for questionable external validity, in the sense that the conclusion may not generalize well to a larger population. The randomized survey experiment is a design tool that can help mitigate this concern, by randomly selecting the experimental units from the target population of interest. However, as pointed out by Morgan and Rubin (2012), chance imbalances often exist in covariate distributions between different treatment groups even under completely randomized experiments. Not surprisingly, such covariate imbalances also occur in randomized survey experiments. Furthermore, the covariate imbalances happen not only between different treatment groups, but also between the sampled experimental units and the overall population of interest. In this paper, we propose a two-stage rerandomization design that can actively avoid undesirable covariate imbalances at both the sampling and treatment assignment stages. We further develop asymptotic theory for rerandomized survey experiments, demonstrating that rerandomization provides better covariate balance, more precise treatment effect estimators, and shorter large-sample confidence intervals. We also propose covariate adjustment to deal with remaining covariate imbalances after rerandomization, showing that it can further improve both the sampling and estimated precision. Our work allows general relationship among covariates at the sampling, treatment assignment and analysis stages, and generalizes both rerandomization in classical randomized experiments (Morgan and Rubin 2012) and rejective sampling in survey sampling (Fuller 2009).


page 1

page 2

page 3

page 4


Sampling-based randomized designs for causal inference under the potential outcomes framework

We establish the inferential properties of the mean-difference estimator...

A general theory of regression adjustment for covariate-adaptive randomization: OLS, Lasso, and beyond

We consider the problem of estimating and inferring treatment effects in...

Rerandomization in 2^K Factorial Experiments

With many pretreatment covariates and treatment factors, the classical f...

Covariate Balancing Based on Kernel Density Estimates for Controlled Experiments

Controlled experiments are widely used in many applications to investiga...

The P-LOOP Estimator: Covariate Adjustment for Paired Experiments

In paired experiments, participants are grouped into pairs with similar ...

Rerandomization and covariate adjustment in split-plot designs

The split-plot design arises from agricultural sciences with experimenta...

Covariate Balancing Methods for Randomized Controlled Trials Are Not Adversarially Robust

The first step towards investigating the effectiveness of a treatment is...