Multilevel Monte-Carlo for Solving POMDPs Online

07/23/2019
by   Marcus Hoerger, et al.
0

Planning under partial obervability is essential for autonomous robots. A principled way to address such planning problems is the Partially Observable Markov Decision Process (POMDP). Although solving POMDPs is computationally intractable, substantial advancements have been achieved in developing approximate POMDP solvers in the past two decades. However, computing robust solutions for systems with complex dynamics remain challenging. Most on-line solvers rely on a large number of forward-simulations and standard Monte-Carlo methods to compute the expected outcomes of actions the robot can perform. For systems with complex dynamics, e.g., those with non-linear dynamics that admit no closed form solution, even a single forward simulation can be prohibitively expensive. Of course, this issue exacerbates for problems with long planning horizons. This paper aims to alleviate the above difficulty. To this end, we propose a new on-line POMDP solver, called Multilevel POMDP Planner (MLPP), that combines the commonly known Monte-Carlo-Tree-Search with the concept of Multilevel Monte-Carlo to speed-up our capability in generating approximately optimal solutions for POMDPs with complex dynamics. Experiments on four different problems of POMDP-based torque control, navigation and grasping indicate that MLPP substantially outperforms state-of-the-art POMDP solvers.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/04/2020

An On-Line POMDP Solver for Continuous Observation Spaces

Planning under partial obervability is essential for autonomous robots. ...
research
05/29/2020

Non-Linearity Measure for POMDP-based Motion Planning

Motion planning under uncertainty is essential for reliable robot operat...
research
06/09/2023

Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments

The design of autonomous agents that can interact effectively with other...
research
02/12/2015

Monte Carlo Planning method estimates planning horizons during interactive social exchange

Reciprocating interactions represent a central feature of all human exch...
research
07/14/2022

Parallel Monte Carlo Tree Search with Batched Rigid-body Simulations for Speeding up Long-Horizon Episodic Robot Planning

We propose a novel Parallel Monte Carlo tree search with Batched Simulat...
research
12/23/2020

Identification of Unexpected Decisions in Partially Observable Monte-Carlo Planning: a Rule-Based Approach

Partially Observable Monte-Carlo Planning (POMCP) is a powerful online a...
research
09/13/2022

Adaptive Discretization using Voronoi Trees for Continuous-Action POMDPs

Solving Partially Observable Markov Decision Processes (POMDPs) with con...

Please sign up or login with your details

Forgot password? Click here to reset