Efficient and Robust Algorithms for Adversarial Linear Contextual Bandits

02/01/2020
by   Gergely Neu, et al.
0

We consider an adversarial variant of the classic K-armed linear contextual bandit problem where the sequence of loss functions associated with each arm are allowed to change without restriction over time. Under the assumption that the d-dimensional contexts are generated i.i.d. at random from a known distributions, we develop computationally efficient algorithms based on the classic Exp3 algorithm. Our first algorithm, RealLinExp3, is shown to achieve a regret guarantee of O(√(KdT)) over T rounds, which matches the best available bound for this problem. Our second algorithm, RobustLinExp3, is shown to be robust to misspecification, in that it achieves a regret bound of O((Kd)^1/3T^2/3) + ε√(d) T if the true reward function is linear up to an additive nonlinear error uniformly bounded in absolute value by ε. To our knowledge, our performance guarantees constitute the very first results on this problem setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/17/2020

Stochastic Bandits with Linear Constraints

We study a constrained contextual linear bandit setting, where the goal ...
11/11/2019

Neural Contextual Bandits with Upper Confidence Bound-Based Exploration

We study the stochastic contextual bandit problem, where the reward is g...
09/07/2022

Dual Instrumental Method for Confounded Kernelized Bandits

The contextual bandit problem is a theoretically justified framework wit...
09/26/2013

Finite-Time Analysis of Kernelised Contextual Bandits

We tackle the problem of online reward maximisation over a large finite ...
06/09/2021

Contextual Recommendations and Low-Regret Cutting-Plane Algorithms

We consider the following variant of contextual linear bandits motivated...
11/09/2021

Practical, Provably-Correct Interactive Learning in the Realizable Setting: The Power of True Believers

We consider interactive learning in the realizable setting and develop a...
10/25/2020

Tractable contextual bandits beyond realizability

Tractable contextual bandit algorithms often rely on the realizability a...