Better Experimental Design by Hybridizing Binary Matching with Imbalance Optimization

by   Abba M. Krieger, et al.

We present a new experimental design procedure that divides a set of experimental units into two groups in order to minimize error in estimating an additive treatment effect. One concern is minimizing error at the experimental design stage is large covariate imbalance between the two groups. Another concern is robustness of design to misspecification in response models. We address both concerns in our proposed design: we first place subjects into pairs using optimal nonbipartite matching, making our estimator robust to complicated non-linear response models. Our innovation is to keep the matched pairs extant, take differences of the covariate values within each matched pair and then we use the greedy switching heuristic of Krieger et al. (2019) or rerandomization on these differences. This latter step greatly reduce covariate imbalance to the rate O_p(n^-4) in the case of one covariate that are uniformly distributed. This rate benefits from the greedy switching heuristic which is O_p(n^-3) and the rate of matching which is O_p(n^-1). Further, our resultant designs are shown to be as random as matching which is robust to unobserved covariates. When compared to previous designs, our approach exhibits significant improvement in the mean squared error of the treatment effect estimator when the response model is nonlinear and performs at least as well when it the response model is linear. Our design procedure is found as a method in the open source R package available on CRAN called GreedyExperimentalDesign.



There are no comments yet.


page 9


Improving the Power of the Randomization Test

We consider the problem of evaluating designs for a two-arm randomized e...

A Discrepancy-Based Design for A/B Testing Experiments

The aim of this paper is to introduce a new design of experiment method ...

Balancing covariates in randomized experiments using the Gram-Schmidt walk

The paper introduces a class of experimental designs that allows experim...

Optimal Rerandomization via a Criterion that Provides Insurance Against Failed Experiments

We present an optimized rerandomization design procedure for a non-seque...

Covariate Balancing Based on Kernel Density Estimates for Controlled Experiments

Controlled experiments are widely used in many applications to investiga...

Nonmyopic and pseudo-nonmyopic approaches to optimal sequential design in the presence of covariates

In sequential experiments, subjects become available for the study over ...

Group-matching algorithms for subjects and items

We consider the problem of constructing matched groups such that the res...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.