Design-Based Inference for Multi-arm Bandits

by   Dae Woong Ham, et al.

Multi-arm bandits are gaining popularity as they enable real-world sequential decision-making across application areas, including clinical trials, recommender systems, and online decision-making. Consequently, there is an increased desire to use the available adaptively collected datasets to distinguish whether one arm was more effective than the other, e.g., which product or treatment was more effective. Unfortunately, existing tools fail to provide valid inference when data is collected adaptively or require many untestable and technical assumptions, e.g., stationarity, iid rewards, bounded random variables, etc. Our paper introduces the design-based approach to inference for multi-arm bandits, where we condition the full set of potential outcomes and perform inference on the obtained sample. Our paper constructs valid confidence intervals for both the reward mean of any arm and the mean reward difference between any arms in an assumption-light manner, allowing the rewards to be arbitrarily distributed, non-iid, and from non-stationary distributions. In addition to confidence intervals, we also provide valid design-based confidence sequences, sequences of confidence intervals that have uniform type-1 error guarantees over time. Confidence sequences allow the agent to perform a hypothesis test as the data arrives sequentially and stop the experiment as soon as the agent is satisfied with the inference, e.g., the mean reward of an arm is statistically significantly higher than a desired threshold.


page 1

page 2

page 3

page 4


Design-Based Confidence Sequences for Anytime-valid Causal Inference

Many organizations run thousands of randomized experiments, or A/B tests...

Inference for Batched Bandits

As bandit algorithms are increasingly utilized in scientific studies, th...

Confidence Intervals for Policy Evaluation in Adaptive Experiments

Adaptive experiments can result in considerable cost savings in multi-ar...

Simultaneous confidence intervals for an extended Koch-Röhmel design in three-arm non-inferiority trials

Three-arm `gold-standard' non-inferiority trials are recommended for ind...

Safe Sequential Testing and Effect Estimation in Stratified Count Data

Sequential decision making significantly speeds up research and is more ...

Li, Li, and Dai's Contribution to the Discussion of "Estimating Means of Bounded Random Variables by Betting" by Waudby-Smith and Aaditya Ramdas

We congratulate Waudby-Smith and Ramdas for their interesting paper <cit...

Comparing Sequential Forecasters

Consider two or more forecasters, each making a sequence of predictions ...

Please sign up or login with your details

Forgot password? Click here to reset