Partially Observable Markov Decision Processes (POMDPs) and Robotics

07/15/2021
by   Hanna Kurniawati, et al.
0

Planning under uncertainty is critical to robotics. The Partially Observable Markov Decision Process (POMDP) is a mathematical framework for such planning problems. It is powerful due to its careful quantification of the non-deterministic effects of actions and partial observability of the states. But precisely because of this, POMDP is notorious for its high computational complexity and deemed impractical for robotics. However, since early 2000, POMDPs solving capabilities have advanced tremendously, thanks to sampling-based approximate solvers. Although these solvers do not generate the optimal solution, they can compute good POMDP solutions that significantly improve the robustness of robotics systems within reasonable computational resources, thereby making POMDPs practical for many realistic robotics problems. This paper presents a review of POMDPs, emphasizing computational issues that have hindered its practicality in robotics and ideas in sampling-based solvers that have alleviated such difficulties, together with lessons learned from applying POMDPs to physical robots.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/20/2015

Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

The focus of this paper is on solving multi-robot planning problems in c...
research
03/19/2021

Knowledge-Based Hierarchical POMDPs for Task Planning

The main goal in task planning is to build a sequence of actions that ta...
research
06/01/2011

Value-Function Approximations for Partially Observable Markov Decision Processes

Partially observable Markov decision processes (POMDPs) provide an elega...
research
04/21/2020

pomdp_py: A Framework to Build and Solve POMDP Problems

In this paper, we present pomdp_py, a general purpose Partially Observab...
research
09/04/2020

Performance Analysis of FEM Solvers on Practical Electromagnetic Problems

The paper presents a comparative analysis of different commercial and ac...
research
12/22/2020

Autonomous sPOMDP Environment Modeling With Partial Model Exploitation

A state space representation of an environment is a classic and yet powe...
research
09/08/2022

Double Q-Learning for Citizen Relocation During Natural Hazards

Natural disasters can cause substantial negative socio-economic impacts ...

Please sign up or login with your details

Forgot password? Click here to reset