Regime Switching Bandits

01/26/2020
by   Xiang Zhou, et al.
6

We study a multi-armed bandit problem where the rewards exhibit regime-switching. Specifically, the distributions of the random rewards generated from all arms depend on a common underlying state modeled as a finite-state Markov chain. The agent does not observe the underlying state and has to learn the unknown transition probability matrix as well as the reward distribution. We propose an efficient learning algorithm for this problem, building on spectral method-of-moments estimations for hidden Markov models and upper confidence bound methods for reinforcement learning. We also establish O(T^2/3√(log T)) bound on the regret of the proposed learning algorithm where T is the unknown horizon. Finally, we conduct numerical experiments to illustrate the effectiveness of the learning algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/08/2021

Sublinear Regret for Learning POMDPs

We study the model-based undiscounted reinforcement learning for partial...
research
03/11/2018

Incentives in the Dark: Multi-armed Bandits for Evolving Users with Unknown Type

Design of incentives or recommendations to users is becoming more common...
research
09/20/2021

Reinforcement Learning for Finite-Horizon Restless Multi-Armed Multi-Action Bandits

We study a finite-horizon restless multi-armed bandit problem with multi...
research
11/13/2020

Active Reinforcement Learning: Observing Rewards at a Cost

Active reinforcement learning (ARL) is a variant on reinforcement learni...
research
07/13/2021

Markov Game with Switching Costs

We study a general Markov game with metric switching costs: in each roun...
research
08/09/2021

Whittle Index for A Class of Restless Bandits with Imperfect Observations

We consider a class of restless bandit problems that finds a broad appli...
research
06/25/2023

Joint Learning of Network Topology and Opinion Dynamics Based on Bandit Algorithms

We study joint learning of network topology and a mixed opinion dynamics...

Please sign up or login with your details

Forgot password? Click here to reset