Adapting to misspecification in contextual bandits with offline regression oracles

by   Sanath Kumar Krishnamurthy, et al.

Computationally efficient contextual bandits are often based on estimating a predictive model of rewards given contexts and arms using past data. However, when the reward model is not well-specified, the bandit algorithm may incur unexpected regret, so recent work has focused on algorithms that are robust to misspecification. We propose a simple family of contextual bandit algorithms that adapt to misspecification error by reverting to a good safe policy when there is evidence that misspecification is causing a regret increase. Our algorithm requires only an offline regression oracle to ensure regret guarantees that gracefully degrade in terms of a measure of the average misspecification level. Compared to prior work, we attain similar regret guarantees, but we do no rely on a master algorithm, and do not require more robust oracles like online or constrained regression oracles (e.g., Foster et al. (2020a); Krishnamurthy et al. (2020)). This allows us to design algorithms for more general function approximation classes.


page 1

page 2

page 3

page 4


Adapting to Misspecification in Contextual Bandits

A major research direction in contextual bandits is to develop algorithm...

Contextual bandits with surrogate losses: Margin bounds and efficient algorithms

We introduce a new family of margin-based regret guarantees for adversar...

Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability

We consider the general (stochastic) contextual bandit problem under the...

Semiparametric Contextual Bandits

This paper studies semiparametric contextual bandits, a generalization o...

Tractable contextual bandits beyond realizability

Tractable contextual bandit algorithms often rely on the realizability a...

On Limited-Memory Subsampling Strategies for Bandits

There has been a recent surge of interest in nonparametric bandit algori...

Sequential Matrix Completion

We propose a novel algorithm for sequential matrix completion in a recom...