DeepAI AI Chat
Log In Sign Up

POMCPOW: An online algorithm for POMDPs with continuous state, action, and observation spaces

09/18/2017
by   Zachary Sunberg, et al.
0

Online solvers for partially observable Markov decision processes have been applied to problems with large discrete state spaces, but continuous state, action, and observation spaces remain a challenge. This paper begins by investigating double progressive widening (DPW) as a solution to this challenge. However, we prove that this modification alone is not sufficient because the belief representations in the search tree collapse to a single particle causing the algorithm to converge to a policy that is suboptimal regardless of the computation time. The main contribution of the paper is to propose a new algorithm, POMCPOW, that incorporates DPW and weighted particle filtering to overcome this deficiency and attack continuous problems. Simulation results show that these modifications allow the algorithm to be successful where previous approaches fail.

READ FULL TEXT
12/23/2022

Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent

Rather than augmenting rewards with penalties for undesired behavior, Co...
11/04/2020

An On-Line POMDP Solver for Continuous Observation Spaces

Planning under partial obervability is essential for autonomous robots. ...
03/16/2017

Scalable Accelerated Decentralized Multi-Robot Policy Search in Continuous Observation Spaces

This paper presents the first ever approach for solving continuous-obser...
02/21/2023

Adaptive Discretization using Voronoi Trees for Continuous POMDPs

Solving continuous Partially Observable Markov Decision Processes (POMDP...
10/10/2019

Sparse tree search optimality guarantees in POMDPs with continuous observation spaces

Partially observable Markov decision processes (POMDPs) with continuous ...

Code Repositories

POMCPOW.jl

Online solver based on Monte Carlo tree search for POMDPs with continuous state, action, and observation spaces.


view repo