DeepAI AI Chat
Log In Sign Up

A Survey of Risk-Aware Multi-Armed Bandits

by   Vincent Y. F. Tan, et al.

In several applications such as clinical trials and financial portfolio optimization, the expected value (or the average reward) does not satisfactorily capture the merits of a drug or a portfolio. In such applications, risk plays a crucial role, and a risk-aware performance measure is preferable, so as to capture losses in the case of adverse events. This survey aims to consolidate and summarise the existing research on risk measures, specifically in the context of multi-armed bandits. We review various risk measures of interest, and comment on their properties. Next, we review existing concentration inequalities for various risk measures. Then, we proceed to defining risk-aware bandit problems, We consider algorithms for the regret minimization setting, where the exploration-exploitation trade-off manifests, as well as the best-arm identification setting, which is a pure exploration problem – both in the context of risk-sensitive measures. We conclude by commenting on persisting challenges and fertile areas for future research.


page 1

page 2

page 3

page 4


Risk-aware Multi-armed Bandits Using Conditional Value-at-Risk

Traditional multi-armed bandit problems are geared towards finding the a...

Risk-aware linear bandits with convex loss

In decision-making problems such as the multi-armed bandit, an agent lea...

Risk-Averse Explore-Then-Commit Algorithms for Finite-Time Bandits

In this paper, we study multi-armed bandit problems in explore-then-comm...

Towards an efficient and risk aware strategy for guiding farmers in identifying best crop management

Identification of best performing fertilizer practices among a set of co...

Risk-Constrained Thompson Sampling for CVaR Bandits

The multi-armed bandit (MAB) problem is a ubiquitous decision-making pro...

Continuous Mean-Covariance Bandits

Existing risk-aware multi-armed bandit models typically focus on risk me...

Multi-Armed Bandits in Brain-Computer Interfaces

The multi-armed bandit (MAB) problem models a decision-maker that optimi...