Thompson Sampling for CVaR Bandits

12/10/2020
by   Dorian Baudry, et al.
0

Risk awareness is an important feature to formulate a variety of real world problems. In this paper we study a multi-arm bandit problem in which the quality of each arm is measured by the Conditional Value at Risk (CVaR) at some level α of the reward distribution. While existing works in this setting mainly focus on Upper Confidence Bound algorithms, we introduce the first Thompson Sampling approaches for CVaR bandits. Building on a recent work by Riou and Honda (2020), we propose α-NPTS for bounded rewards and α-Multinomial-TS for multinomial distributions. We provide a novel lower bound on the CVaR regret which extends the concept of asymptotic optimality to CVaR bandits and prove that α-Multinomial-TS is the first algorithm to achieve this lower bound. Finally, we demonstrate empirically the benefit of Thompson Sampling approaches over their UCB counterparts.

READ FULL TEXT
research
06/15/2021

Thompson Sampling for Unimodal Bandits

In this paper, we propose a Thompson Sampling algorithm for unimodal ban...
research
10/19/2021

Batched Lipschitz Bandits

In this paper, we study the batched Lipschitz bandit problem, where the ...
research
05/08/2018

Profitable Bandits

Originally motivated by default risk management applications, this paper...
research
10/07/2020

Effects of Model Misspecification on Bayesian Bandits: Case Studies in UX Optimization

Bayesian bandits using Thompson Sampling have seen increasing success in...
research
07/17/2020

Bandits for BMO Functions

We study the bandit problem where the underlying expected reward is a Bo...
research
11/18/2021

From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Bandits

The stochastic multi-arm bandit problem has been extensively studied und...
research
08/23/2023

On Uniformly Optimal Algorithms for Best Arm Identification in Two-Armed Bandits with Fixed Budget

We study the problem of best-arm identification with fixed budget in sto...

Please sign up or login with your details

Forgot password? Click here to reset