Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts

02/02/2022
by   Hongju Park, et al.
0

Contextual bandits are widely-used in the study of learning-based control policies for finite action spaces. While the problem is well-studied for bandits with perfectly observed context vectors, little is known about the case of imperfectly observed contexts. For this setting, existing approaches are inapplicable and new conceptual and technical frameworks are required. We present an implementable posterior sampling algorithm for bandits with imperfect context observations and study its performance for learning optimal decisions. The provided numerical results relate the performance of the algorithm to different quantities of interest including the number of arms, dimensions, observation matrices, posterior rescaling factors, and signal-to-noise ratios. In general, the proposed algorithm exposes efficiency in learning from the noisy imperfect observations and taking actions accordingly. Enlightening understandings the analyses provide as well as interesting future directions it points to, are discussed as well.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/10/2022

Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations

Contextual bandits are canonical models for sequential decision-making u...
research
10/23/2021

Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Bandits

Contextual multi-armed bandits are classical models in reinforcement lea...
research
12/10/2020

Adversarial Linear Contextual Bandits with Graph-Structured Side Observations

This paper studies the adversarial graphical contextual bandits, a varia...
research
05/18/2018

PG-TS: Improved Thompson Sampling for Logistic Contextual Bandits

We address the problem of regret minimization in logistic contextual ban...
research
04/25/2016

Double Thompson Sampling for Dueling Bandits

In this paper, we propose a Double Thompson Sampling (D-TS) algorithm fo...
research
07/15/2019

A Dimension-free Algorithm for Contextual Continuum-armed Bandits

In contextual continuum-armed bandits, the contexts x and the arms y are...
research
04/20/2021

Data Envelopment Analysis models with imperfect knowledge of input and output values: An application to Portuguese public hospitals

Assessing the technical efficiency of a set of observations requires tha...

Please sign up or login with your details

Forgot password? Click here to reset