Nonparametric Contextual Bandits in an Unknown Metric Space

08/03/2019
by   Nirandika Wanigasekara, et al.
6

Consider a nonparametric contextual multi-arm bandit problem where each arm a ∈ [K] is associated to a nonparametric reward function f_a: [0,1] →R mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward functions, e.g. finite types or smooth with respect to an unknown metric space. We present a novel algorithm which learns data-driven similarities amongst the arms, in order to implement adaptive partitioning of the context-arm space for more efficient learning. We provide regret bounds along with simulations that highlight the algorithm's dependence on the local geometry of the reward functions.

READ FULL TEXT

page 8

page 29

page 30

page 31

12/17/2021

Learning in Restless Bandits under Exogenous Global Markov Process

We consider an extension to the restless multi-armed bandit (RMAB) probl...
09/08/2022

A Nonparametric Contextual Bandit with Arm-level Eligibility Control for Customer Service Routing

Amazon Customer Service provides real-time support for millions of custo...
07/17/2020

Bandits for BMO Functions

We study the bandit problem where the underlying expected reward is a Bo...
02/12/2018

Policy Gradients for Contextual Bandits

We study a generalized contextual-bandits problem, where there is a stat...
04/19/2018

Exploring Partially Observed Networks with Nonparametric Bandits

Real-world networks such as social and communication networks are too la...
10/05/2021

Contextual Combinatorial Volatile Bandits via Gaussian Processes

We consider a contextual bandit problem with a combinatorial action set ...
10/11/2019

Privacy-Preserving Contextual Bandits

Contextual bandits are online learners that, given an input, select an a...