DeepAI AI Chat
Log In Sign Up

Adaptive A/B Tests and Simultaneous Treatment Parameter Optimization

10/13/2022
by   Yuhang Wu, et al.
0

Constructing asymptotically valid confidence intervals through a valid central limit theorem is crucial for A/B tests, where a classical goal is to statistically assert whether a treatment plan is significantly better than a control plan. In some emerging applications for online platforms, the treatment plan is not a single plan, but instead encompasses an infinite continuum of plans indexed by a continuous treatment parameter. As such, the experimenter not only needs to provide valid statistical inference, but also desires to effectively and adaptively find the optimal choice of value for the treatment parameter to use for the treatment plan. However, we find that classical optimization algorithms, despite of their fast convergence rates under convexity assumptions, do not come with a central limit theorem that can be used to construct asymptotically valid confidence intervals. We fix this issue by providing a new optimization algorithm that on one hand maintains the same fast convergence rate and on the other hand permits the establishment of a valid central limit theorem. We discuss practical implementations of the proposed algorithm and conduct numerical experiments to illustrate the theoretical findings.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/23/2023

Communication-Efficient Distributed Estimation and Inference for Cox's Model

Motivated by multi-center biomedical studies that cannot share individua...
03/11/2021

Doubly robust confidence sequences for sequential causal inference

This paper derives time-uniform confidence sequences (CS) for causal eff...
08/31/2020

Design and Analysis of Switchback Experiments

In switchback experiments, a firm sequentially exposes an experimental u...
01/05/2023

Statistical Inference and A/B Testing for First-Price Pacing Equilibria

We initiate the study of statistical inference and A/B testing for first...
02/05/2021

Online Statistical Inference for Gradient-free Stochastic Optimization

As gradient-free stochastic optimization gains emerging attention for a ...
03/08/2023

Inference on Optimal Dynamic Policies via Softmax Approximation

Estimating optimal dynamic policies from offline data is a fundamental p...
02/10/2021

Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithm

As machine learning models are deployed in critical applications, it bec...