Log In Sign Up

Finite-Time Analysis of Kernelised Contextual Bandits

by   Michal Valko, et al.

We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward is an arbitrary linear function of the contexts' images in the related reproducing kernel Hilbert space (RKHS). We propose KernelUCB, a kernelised UCB algorithm, and give a cumulative regret bound through a frequentist analysis. For contextual bandits, the related algorithm GP-UCB turns out to be a special case of our algorithm, and our finite-time analysis improves the regret bound of GP-UCB for the agnostic case, both in the terms of the kernel-dependent quantity and the RKHS norm of the reward function. Moreover, for the linear kernel, our regret bound matches the lower bound for contextual linear bandits.


page 1

page 2

page 3

page 4


Factored Bandits

We introduce the factored bandits model, which is a framework for learni...

Efficient Kernel UCB for Contextual Bandits

In this paper, we tackle the computational efficiency of kernelized UCB ...

Graph Neural Network Bandits

We consider the bandit optimization problem with the reward function def...

Making the most of your day: online learning for optimal allocation of time

We study online learning for optimal allocation when the resource to be ...

Optimal Contextual Bandits with Knapsacks under Realizibility via Regression Oracles

We study the stochastic contextual bandit with knapsacks (CBwK) problem,...

Online learning with kernel losses

We present a generalization of the adversarial linear bandits framework,...

Efficient and Robust Algorithms for Adversarial Linear Contextual Bandits

We consider an adversarial variant of the classic K-armed linear context...