DeepAI AI Chat
Log In Sign Up

Neural Contextual Bandits without Regret

07/07/2021
by   Parnian Kassraie, et al.
0

Contextual bandits are a rich model for sequential decision making given side information, with important applications, e.g., in recommender systems. We propose novel algorithms for contextual bandits harnessing neural networks to approximate the unknown reward function. We resolve the open problem of proving sublinear regret bounds in this setting for general context sequences, considering both fully-connected and convolutional networks. To this end, we first analyze NTK-UCB, a kernelized bandit optimization algorithm employing the Neural Tangent Kernel (NTK), and bound its regret in terms of the NTK maximum information gain γ_T, a complexity parameter capturing the difficulty of learning. Our bounds on γ_T for the NTK may be of independent interest. We then introduce our neural network based algorithm NN-UCB, and show that its regret closely tracks that of NTK-UCB. Under broad non-parametric assumptions about the reward function, our approach converges to the optimal policy at a 𝒪̃(T^-1/2d) rate, where d is the dimension of the context.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/05/2019

Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes

We study a nonparametric contextual bandit problem where the expected re...
07/13/2022

Graph Neural Network Bandits

We consider the bandit optimization problem with the reward function def...
10/19/2021

Regret Minimization in Isotonic, Heavy-Tailed Contextual Bandits via Adaptive Confidence Bands

In this paper we initiate a study of non parametric contextual bandits u...
05/31/2022

Provably and Practically Efficient Neural Contextual Bandits

We consider the neural contextual bandit problem. In contrast to the exi...
02/16/2021

Making the most of your day: online learning for optimal allocation of time

We study online learning for optimal allocation when the resource to be ...
11/05/2021

An Empirical Study of Neural Kernel Bandits

Neural bandits have enabled practitioners to operate efficiently on prob...
01/31/2023

Improved Algorithms for Multi-period Multi-class Packing Problems with Bandit Feedback

We consider the linear contextual multi-class multi-period packing probl...