Sequential Decision Making under Uncertainty with Dynamic Resource Constraints

04/18/2019
by   Kesav Kaza, et al.
0

This paper studies a class of constrained restless multi-armed bandits. The constraints are in the form of time varying availability of arms. This variation can be either stochastic or semi-deterministic. A fixed number of arms can be chosen to be played in each decision interval. The play of each arm yields a state dependent reward. The current states of arms are partially observable through binary feedback signals from arms that are played. The current availability of arms is fully observable. The objective is to maximize long term cumulative reward. The uncertainty about future availability of arms along with partial state information makes this objective challenging. This optimization problem is analyzed using Whittle's index policy. To this end, a constrained restless single-armed bandit is studied. It is shown to admit a threshold-type optimal policy, and is also indexable. An algorithm to compute Whittle's index is presented. Further, upper bounds on the value function are derived in order to estimate the degree of sub-optimality of various solutions. The simulation study compares the performance of Whittle's index, modified Whittle's index and myopic policies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/04/2018

Lazy Restless Bandits for Decision Making with Limited Observation Capability: Applications in Wireless Networks

In this work we formulate the problem of restless multi-armed bandits wi...
research
04/30/2023

Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policy

We consider finite state restless multi-armed bandit problem. The decisi...
research
03/15/2016

Optimal Sensing via Multi-armed Bandit Relaxations in Mixed Observability Domains

Sequential decision making under uncertainty is studied in a mixed obser...
research
12/09/2022

Networked Restless Bandits with Positive Externalities

Restless multi-armed bandits are often used to model budget-constrained ...
research
07/25/2021

Restless Bandits with Many Arms: Beating the Central Limit Theorem

We consider finite-horizon restless bandits with multiple pulls per peri...
research
03/29/2022

Near-optimality for infinite-horizon restless bandits with many arms

Restless bandits are an important class of problems with applications in...
research
02/22/2017

Approximations of the Restless Bandit Problem

The multi-armed restless bandit problem is studied in the case where the...

Please sign up or login with your details

Forgot password? Click here to reset