Rejective Sampling, Rerandomization and Regression Adjustment in Survey Experiments

09/21/2021
by   Zihao Yang, et al.
0

Classical randomized experiments, equipped with randomization-based inference, provide assumption-free inference for treatment effects. They have been the gold standard for drawing causal inference and provide excellent internal validity. However, they have also been criticized for questionable external validity, in the sense that the conclusion may not generalize well to a larger population. The randomized survey experiment is a design tool that can help mitigate this concern, by randomly selecting the experimental units from the target population of interest. However, as pointed out by Morgan and Rubin (2012), chance imbalances often exist in covariate distributions between different treatment groups even under completely randomized experiments. Not surprisingly, such covariate imbalances also occur in randomized survey experiments. Furthermore, the covariate imbalances happen not only between different treatment groups, but also between the sampled experimental units and the overall population of interest. In this paper, we propose a two-stage rerandomization design that can actively avoid undesirable covariate imbalances at both the sampling and treatment assignment stages. We further develop asymptotic theory for rerandomized survey experiments, demonstrating that rerandomization provides better covariate balance, more precise treatment effect estimators, and shorter large-sample confidence intervals. We also propose covariate adjustment to deal with remaining covariate imbalances after rerandomization, showing that it can further improve both the sampling and estimated precision. Our work allows general relationship among covariates at the sampling, treatment assignment and analysis stages, and generalizes both rerandomization in classical randomized experiments (Morgan and Rubin 2012) and rejective sampling in survey sampling (Fuller 2009).

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/05/2018

Sampling-based randomized designs for causal inference under the potential outcomes framework

We establish the inferential properties of the mean-difference estimator...
09/08/2020

Designing Transportable Experiments

We consider the problem of designing a randomized experiment on a source...
08/12/2020

Covariate Balancing Based on Kernel Density Estimates for Controlled Experiments

Controlled experiments are widely used in many applications to investiga...
12/28/2018

Rerandomization in 2^K Factorial Experiments

With many pretreatment covariates and treatment factors, the classical f...
05/21/2019

The P-LOOP Estimator: Covariate Adjustment for Paired Experiments

In paired experiments, participants are grouped into pairs with similar ...
09/06/2021

Rerandomization with Diminishing Covariate Imbalance and Diverging Number of Covariates

Completely randomized experiments have been the gold standard for drawin...
04/23/2018

Is My Matched Dataset As-If Randomized, More, Or Less? Unifying the Design and Analysis of Observational Studies

Matching alleviates the problem of covariate imbalance in observational ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.