DeepAI AI Chat
Log In Sign Up

Adaptive A/B Tests and Simultaneous Treatment Parameter Optimization

by   Yuhang Wu, et al.

Constructing asymptotically valid confidence intervals through a valid central limit theorem is crucial for A/B tests, where a classical goal is to statistically assert whether a treatment plan is significantly better than a control plan. In some emerging applications for online platforms, the treatment plan is not a single plan, but instead encompasses an infinite continuum of plans indexed by a continuous treatment parameter. As such, the experimenter not only needs to provide valid statistical inference, but also desires to effectively and adaptively find the optimal choice of value for the treatment parameter to use for the treatment plan. However, we find that classical optimization algorithms, despite of their fast convergence rates under convexity assumptions, do not come with a central limit theorem that can be used to construct asymptotically valid confidence intervals. We fix this issue by providing a new optimization algorithm that on one hand maintains the same fast convergence rate and on the other hand permits the establishment of a valid central limit theorem. We discuss practical implementations of the proposed algorithm and conduct numerical experiments to illustrate the theoretical findings.


page 1

page 2

page 3

page 4


Communication-Efficient Distributed Estimation and Inference for Cox's Model

Motivated by multi-center biomedical studies that cannot share individua...

Doubly robust confidence sequences for sequential causal inference

This paper derives time-uniform confidence sequences (CS) for causal eff...

Design and Analysis of Switchback Experiments

In switchback experiments, a firm sequentially exposes an experimental u...

Statistical Inference and A/B Testing for First-Price Pacing Equilibria

We initiate the study of statistical inference and A/B testing for first...

Online Statistical Inference for Gradient-free Stochastic Optimization

As gradient-free stochastic optimization gains emerging attention for a ...

Inference on Optimal Dynamic Policies via Softmax Approximation

Estimating optimal dynamic policies from offline data is a fundamental p...

Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithm

As machine learning models are deployed in critical applications, it bec...