Optimal Best Markovian Arm Identification with Fixed Confidence

12/02/2019
by   Vrettos Moulos, et al.
0

We give a complete characterization of the sampling complexity of best Markovian arm identification in one-parameter Markovian bandit models. We derive instance specific nonasymptotic and asymptotic lower bounds which generalize those of the IID setting. We analyze the Track-and-Stop strategy, initially proposed for the IID setting, and we prove that asymptotically it is at most a factor of four apart from the lower bound. Our one-parameter Markovian bandit model is based on the notion of an exponential family of stochastic matrices for which we establish many useful properties. For the analysis of the Track-and-Stop strategy we derive a novel concentration inequality for Markov chains that may be of interest in its own right.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/15/2016

Optimal Best Arm Identification with Fixed Confidence

We give a complete characterization of the complexity of best-arm identi...
research
06/26/2021

The Role of Contextual Information in Best Arm Identification

We study the best-arm identification problem with fixed confidence when ...
research
11/15/2022

Bayesian Fixed-Budget Best-Arm Identification

Fixed-budget best-arm identification (BAI) is a bandit problem where the...
research
05/09/2019

Non-Asymptotic Sequential Tests for Overlapping Hypotheses and application to near optimal arm identification in bandit models

In this paper, we study sequential testing problems with overlapping hyp...
research
02/16/2017

The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime

We propose a novel technique for analyzing adaptive sampling called the ...
research
04/14/2022

Measurement-based Admission Control in Sliced Networks: A Best Arm Identification Approach

In sliced networks, the shared tenancy of slices requires adaptive admis...
research
05/27/2021

A Non-asymptotic Approach to Best-Arm Identification for Gaussian Bandits

We propose a new strategy for best-arm identification with fixed confide...

Please sign up or login with your details

Forgot password? Click here to reset