To update or not to update? Delayed Nonparametric Bandits with Randomized Allocation

05/26/2020
by   Sakshi Arya, et al.
0

Delayed rewards problem in contextual bandits has been of interest in various practical settings. We study randomized allocation strategies and provide an understanding on how the exploration-exploitation tradeoff is affected by delays in observing the rewards. In randomized strategies, the extent of exploration-exploitation is controlled by a user-determined exploration probability sequence. In the presence of delayed rewards, one may choose between using the original exploration sequence that updates at every time point or update the sequence only when a new reward is observed, leading to two competing strategies. In this work, we show that while both strategies may lead to strong consistency in allocation, the property holds for a wider scope of situations for the latter. However, for finite sample performance, we illustrate that both strategies have their own advantages and disadvantages, depending on the severity of the delay and underlying reward generating mechanisms.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/03/2019

Randomized Allocation with Nonparametric Estimation for Contextual Multi-Armed Bandits with Delayed Rewards

We study a multi-armed bandit problem with covariates in a setting where...
research
11/14/2019

Contextual Bandits Evolving Over Finite Time

Contextual bandits have the same exploration-exploitation trade-off as s...
research
05/04/2018

BelMan: Bayesian Bandits on the Belief--Reward Manifold

We propose a generic, Bayesian, information geometric approach to the ex...
research
03/11/2020

Delay-Adaptive Learning in Generalized Linear Contextual Bandits

In this paper, we consider online learning in generalized linear context...
research
09/16/2016

Exploration Potential

We introduce exploration potential, a quantity that measures how much a ...
research
08/10/2022

Increasing Students' Engagement to Reminder Emails Through Multi-Armed Bandits

Conducting randomized experiments in education settings raises the quest...
research
04/15/2011

Polyethism in a colony of artificial ants

We explore self-organizing strategies for role assignment in a foraging ...

Please sign up or login with your details

Forgot password? Click here to reset