A Best-of-Both-Worlds Algorithm for Bandits with Delayed Feedback
share
We present a modified tuning of the algorithm of Zimmert and Seldin [2020] for adversarial multiarmed bandits with delayed feedback, which in addition to the minimax optimal adversarial regret guarantee shown by Zimmert and Seldin simultaneously achieves a near-optimal regret guarantee in the stochastic setting with fixed delays. Specifically, the adversarial regret guarantee is 𝒪(√(TK) + √(dTlog K)), where T is the time horizon, K is the number of arms, and d is the fixed delay, whereas the stochastic regret guarantee is 𝒪(∑_i ≠ i^*(1/Δ_ilog(T) + d/Δ_ilog K) + d K^1/3log K), where Δ_i are the suboptimality gaps. We also present an extension of the algorithm to the case of arbitrary delays, which is based on an oracle knowledge of the maximal delay d_max and achieves 𝒪(√(TK) + √(Dlog K) + d_maxK^1/3log K) regret in the adversarial regime, where D is the total delay, and 𝒪(∑_i ≠ i^*(1/Δ_ilog(T) + σ_max/Δ_ilog K) + d_maxK^1/3log K) regret in the stochastic regime, where σ_max is the maximal number of outstanding observations. Finally, we present a lower bound that matches regret upper bound achieved by the skipping technique of Zimmert and Seldin [2020] in the adversarial setting.
READ FULL TEXT