Improving the Power of the Randomization Test

by   Abba M. Krieger, et al.

We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation assumes a response that is linear in one observed covariate, an unobserved component and an additive treatment effect where the only randomness comes from the treatment allocations. It is well-known that the power depends on the allocations' imbalance in the observed covariate and this is the reason for the classic restricted designs such as rerandomization. We show that power is also affected by two other design choices: the number of allocations in the design and the degree of linear dependence among the allocations. We prove that the more allocations, the higher the power and the lower the variability in the power. Designs that feature greater independence of allocations are also shown to have higher performance. Our theoretical findings and extensive simulation studies imply that the designs with the highest power provide thousands of highly independent allocations that each provide nominal imbalance in the observed covariates. These high powered designs exhibit less randomization than complete randomization and more randomization than recently proposed designs based on numerical optimization. Model choices for a practicing experimenter are rerandomization and greedy pair switching, where both outperform complete randomization and numerical optimization. The tradeoff we find also provides a means to specify the imbalance threshold parameter when rerandomizing.



There are no comments yet.


page 8

page 11

page 29

page 31


Randomization Tests that Condition on Non-Categorical Covariate Balance

A benefit of randomized experiments is that covariate distributions of t...

Better Experimental Design by Hybridizing Binary Matching with Imbalance Optimization

We present a new experimental design procedure that divides a set of exp...

Harmonizing Fully Optimal Designs with Classic Randomization in Fixed Trial Experiments

There is a movement in design of experiments away from the classic rando...

Properties of restricted randomization with implications for experimental design

Recently, there as been an increasing interest in the use of heavily res...

Optimal Rerandomization via a Criterion that Provides Insurance Against Failed Experiments

We present an optimized rerandomization design procedure for a non-seque...

Constrained randomization and statistical inference for multi-arm parallel cluster randomized controlled trials

Cluster randomized controlled trials (cRCTs) are designed to evaluate in...

On the Theory of Covariate-Adaptive Designs

Pocock and Simon's marginal procedure (Pocock and Simon, 1975) is often ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.