A Tractable POMDP for a Class of Sequencing Problems

01/10/2013
by   Paat Rusmevichientong, et al.
0

We consider a partially observable Markov decision problem (POMDP) that models a class of sequencing problems. Although POMDPs are typically intractable, our formulation admits tractable solution. Instead of maintaining a value function over a high-dimensional set of belief states, we reduce the state space to one of smaller dimension, in which grid-based dynamic programming techniques are effective. We develop an error bound for the resulting approximation, and discuss an application of the model to a problem in targeted advertising.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/06/2021

HJB-RBF based approach for the control of PDEs

Semi-lagrangian schemes for discretization of the dynamic programming pr...
research
07/11/2012

Dynamic Programming for Structured Continuous Markov Decision Problems

We describe an approach for exploiting structure in Markov Decision Proc...
research
10/22/2022

B^3RTDP: A Belief Branch and Bound Real-Time Dynamic Programming Approach to Solving POMDPs

Partially Observable Markov Decision Processes (POMDPs) offer a promisin...
research
06/30/2011

Finding Approximate POMDP solutions Through Belief Compression

Standard value function approaches to finding policies for Partially Obs...
research
01/06/2019

Large-Scale Markov Decision Problems via the Linear Programming Dual

We consider the problem of controlling a fully specified Markov decision...
research
03/13/2019

Compact Representation of Value Function in Partially Observable Stochastic Games

Value methods for solving stochastic games with partial observability mo...
research
02/15/2019

Bi-directional Value Learning for Risk-aware Planning Under Uncertainty

Decision-making under uncertainty is a crucial ability for autonomous sy...

Please sign up or login with your details

Forgot password? Click here to reset