Online Planning Algorithms for POMDPs

01/15/2014
by   Stéphane Ross, et al.
0

Partially Observable Markov Decision Processes (POMDPs) provide a rich framework for sequential decision-making under uncertainty in stochastic domains. However, solving a POMDP is often intractable except for small problems due to their complexity. Here, we focus on online approaches that alleviate the computational complexity by computing good local policies at each decision step during the execution. Online algorithms generally consist of a lookahead search to find the best action to execute at each time step in an environment. Our objectives here are to survey the various existing online POMDP methods, analyze their properties and discuss their advantages and disadvantages; and to thoroughly evaluate these online approaches in different environments under various metrics (return, error bound reduction, lower bound improvement). Our experimental results indicate that state-of-the-art online heuristic search methods can handle large POMDP domains efficiently.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2011

Value-Function Approximations for Partially Observable Markov Decision Processes

Partially observable Markov decision processes (POMDPs) provide an elega...
research
10/16/2012

FHHOP: A Factored Hybrid Heuristic Online Planning Algorithm for Large POMDPs

Planning in partially observable Markov decision processes (POMDPs) rema...
research
02/27/2013

Integrating Planning and Execution in Stochastic Domains

We investigate planning in time-critical domains represented as Markov D...
research
09/19/2023

Safe POMDP Online Planning via Shielding

Partially observable Markov decision processes (POMDPs) have been widely...
research
10/16/2012

Scaling Up Decentralized MDPs Through Heuristic Search

Decentralized partially observable Markov decision processes (Dec-POMDPs...
research
08/19/2019

Evaluating Hierarchies through A Partially Observable Markov Decision Processes Methodology

Hierarchical clustering has been shown to be valuable in many scenarios,...
research
06/03/2022

Robotic Planning under Uncertainty in Spatiotemporal Environments in Expeditionary Science

In the expeditionary sciences, spatiotemporally varying environments – h...

Please sign up or login with your details

Forgot password? Click here to reset