Effects of Model Misspecification on Bayesian Bandits: Case Studies in UX Optimization

10/07/2020
by   Mack Sweeney, et al.
0

Bayesian bandits using Thompson Sampling have seen increasing success in recent years. Yet existing value models (of rewards) are misspecified on many real-world problem. We demonstrate this on the User Experience Optimization (UXO) problem, providing a novel formulation as a restless, sleeping bandit with unobserved confounders plus optional stopping. Our case studies show how common misspecifications can lead to sub-optimal rewards, and we provide model extensions to address these, along with a scientific model building process practitioners can adopt or adapt to solve their own unique problems. To our knowledge, this is the first study showing the effects of overdispersion on bandit explore/exploit efficacy, tying the common notions of under- and over-confidence to over- and under-exploration, respectively. We also present the first model to exploit cointegration in a restless bandit, demonstrating that finite regret and fast and consistent optional stopping are possible by moving beyond simpler windowing, discounting, and drift models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/24/2018

New Insights into Bootstrapping for Bandits

We investigate the use of bootstrapping in the bandit setting. We first ...
research
12/10/2020

Thompson Sampling for CVaR Bandits

Risk awareness is an important feature to formulate a variety of real wo...
research
06/16/2021

Reinforcement Learning for Markovian Bandits: Is Posterior Sampling more Scalable than Optimism?

We study learning algorithms for the classical Markovian bandit problem ...
research
05/12/2019

On the Performance of Thompson Sampling on Logistic Bandits

We study the logistic bandit, in which rewards are binary with success p...
research
05/25/2021

Bias-Robust Bayesian Optimization via Dueling Bandit

We consider Bayesian optimization in settings where observations can be ...
research
08/09/2014

Bandit Algorithms for Tree Search

Bandit based methods for tree search have recently gained popularity whe...
research
02/02/2023

Practical Bandits: An Industry Perspective

The bandit paradigm provides a unified modeling framework for problems t...

Please sign up or login with your details

Forgot password? Click here to reset