DeepAI AI Chat
Log In Sign Up

Almost Optimal Batch-Regret Tradeoff for Batch Linear Contextual Bandits

10/15/2021
by   Zihan Zhang, et al.
5

We study the optimal batch-regret tradeoff for batch linear contextual bandits. For any batch number M, number of actions K, time horizon T, and dimension d, we provide an algorithm and prove its regret guarantee, which, due to technical reasons, features a two-phase expression as the time horizon T grows. We also prove a lower bound theorem that surprisingly shows the optimality of our two-phase regret upper bound (up to logarithmic factors) in the full range of the problem parameters, therefore establishing the exact batch-regret tradeoff. Compared to the recent work <cit.> which showed that M = O(loglog T) batches suffice to achieve the asymptotically minimax-optimal regret without the batch constraints, our algorithm is simpler and easier for practical implementation. Furthermore, our algorithm achieves the optimal regret for all T ≥ d, while <cit.> requires that T greater than an unrealistically large polynomial of d. Along our analysis, we also prove a new matrix concentration inequality with dependence on their dynamic upper bounds, which, to the best of our knowledge, is the first of its kind in literature and maybe of independent interest.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/27/2020

Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits

We study the problem of dynamic batch learning in high-dimensional spars...
03/30/2019

Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits

We study the linear contextual bandit problem with finite action sets. W...
02/02/2023

Stochastic Contextual Bandits with Long Horizon Rewards

The growing interest in complex decision-making and language modeling pr...
04/14/2020

Sequential Batch Learning in Finite-Action Linear Contextual Bandits

We study the sequential batch learning problem in linear contextual band...
03/05/2020

Generalized Policy Elimination: an efficient algorithm for Nonparametric Contextual Bandits

We propose the Generalized Policy Elimination (GPE) algorithm, an oracle...
02/12/2021

The Symmetry between Arms and Knapsacks: A Primal-Dual Approach for Bandits with Knapsacks

In this paper, we study the bandits with knapsacks (BwK) problem and dev...
04/27/2015

Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits

We study contextual bandits with budget and time constraints, referred t...