Nonstochastic Multiarmed Bandits with Unrestricted Delays

06/03/2019
by   Tobias Sommer Thune, et al.
0

We investigate multiarmed bandits with delayed feedback, where the delays need neither be identical nor bounded. We first prove that the "delayed" Exp3 achieves the O(√((KT + D) K)) regret bound conjectured by Cesa-Bianchi et al. [2016], in the case of variable, but bounded delays. Here, K is the number of actions and D is the total delay over T rounds. We then introduce a new algorithm that lifts the requirement of bounded delays by using a wrapper that skips rounds with excessively large delays. The new algorithm maintains the same regret bound, but similar to its predecessor requires prior knowledge of D and T. For this algorithm we then construct a novel doubling scheme that forgoes this requirement under the assumption that the delays are available at action time (rather than at loss observation time). This assumption is satisfied in a broad range of applications, including interaction with servers and service providers. The resulting oracle regret bound is of order _β (|S_β|+β K + (KT + D_β)/β), where |S_β| is the number of observations with delay exceeding β, and D_β is the total delay of observations with delay below β. The bound relaxes to O(√((KT + D) K)), but we also provide examples where D_β≪ D and the oracle bound has a polynomially better dependence on the problem parameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/14/2019

An Optimal Algorithm for Adversarial Bandits with Arbitrary Delays

We propose a new algorithm for adversarial multi-armed bandits with unre...
research
08/21/2023

An Improved Best-of-both-worlds Algorithm for Bandits with Delayed Feedback

We propose a new best-of-both-worlds algorithm for bandits with variably...
research
06/29/2022

A Best-of-Both-Worlds Algorithm for Bandits with Delayed Feedback

We present a modified tuning of the algorithm of Zimmert and Seldin [202...
research
03/08/2021

No Discounted-Regret Learning in Adversarial Bandits with Delays

Consider a player that in each round t out of T rounds chooses an action...
research
10/12/2020

Adapting to Delays and Data in Adversarial Multi-Armed Bandits

We consider the adversarial multi-armed bandit problem under delayed fee...
research
11/02/2021

Nonstochastic Bandits and Experts with Arm-Dependent Delays

We study nonstochastic bandits and experts in a delayed setting where de...
research
02/01/2023

Delayed Feedback in Kernel Bandits

Black box optimisation of an unknown function from expensive and noisy e...

Please sign up or login with your details

Forgot password? Click here to reset