Value-Directed Sampling Methods for POMDPs

01/10/2013
by   Pascal Poupart, et al.
0

We consider the problem of approximate belief-state monitoring using particle filtering for the purposes of implementing a policy for a partially-observable Markov decision process (POMDP). While particle filtering has become a widely-used tool in AI for monitoring dynamical systems, rather scant attention has been paid to their use in the context of decision making. Assuming the existence of a value function, we derive error bounds on decision quality associated with filtering using importance sampling. We also describe an adaptive procedure that can be used to dynamically determine the number of samples required to meet specific error bounds. Empirical evidence is offered supporting this technique as a profitable means of directing sampling effort where it is needed to distinguish policies.

READ FULL TEXT

page 1

page 2

page 3

page 5

page 6

page 7

page 8

page 9

research
01/16/2013

Value-Directed Belief State Approximation for POMDPs

We consider the problem belief-state monitoring for the purposes of impl...
research
06/10/2020

When is Particle Filtering Efficient for POMDP Sequential Planning?

Particle filtering is a popular method for inferring latent states in st...
research
01/05/2021

Improving Training Result of Partially Observable Markov Decision Process by Filtering Beliefs

In this study I proposed a filtering beliefs method for improving perfor...
research
07/24/2022

Towards Using Fully Observable Policies for POMDPs

Partially Observable Markov Decision Process (POMDP) is a framework appl...
research
12/12/2012

Factored Particles for Scalable Monitoring

Exact monitoring in dynamic Bayesian networks is intractable, so approxi...
research
05/03/2021

Homotopy Sampling, with an Application to Particle Filters

We propose a homotopy sampling procedure, loosely based on importance sa...
research
01/10/2013

Vector-space Analysis of Belief-state Approximation for POMDPs

We propose a new approach to value-directed belief state approximation f...

Please sign up or login with your details

Forgot password? Click here to reset