Stochastic Finite State Control of POMDPs with LTL Specifications

01/21/2020
by   Mohamadreza Ahmadi, et al.
0

Partially observable Markov decision processes (POMDPs) provide a modeling framework for autonomous decision making under uncertainty and imperfect sensing, e.g. robot manipulation and self-driving cars. However, optimal control of POMDPs is notoriously intractable. This paper considers the quantitative problem of synthesizing sub-optimal stochastic finite state controllers (sFSCs) for POMDPs such that the probability of satisfying a set of high-level specifications in terms of linear temporal logic (LTL) formulae is maximized. We begin by casting the latter problem into an optimization and use relaxations based on the Poisson equation and McCormick envelopes. Then, we propose an stochastic bounded policy iteration algorithm, leading to a controlled growth in sFSC size and an any time algorithm, where the performance of the controller improves with successive iterations, but can be stopped by the user based on time or memory considerations. We illustrate the proposed method by a robot navigation case study.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/27/2019

Risk-Averse Planning Under Uncertainty

We consider the problem of designing policies for partially observable M...
research
05/24/2023

Optimal Control of Logically Constrained Partially Observable and Multi-Agent Markov Decision Processes

Autonomous systems often have logical constraints arising, for example, ...
research
02/05/2020

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been e...
research
09/24/2020

Robust Finite-State Controllers for Uncertain POMDPs

Uncertain partially observable Markov decision processes (uPOMDPs) allow...
research
09/21/2022

Partially Observable Markov Decision Processes in Robotics: A Survey

Noisy sensing, imperfect control, and environment changes are defining c...
research
06/13/2012

Sparse Stochastic Finite-State Controllers for POMDPs

Bounded policy iteration is an approach to solving infinite-horizon POMD...
research
02/23/2018

Control and Sensing Co-design

Linear-Quadratic-Gaussian (LQG) control is concerned with the design of ...

Please sign up or login with your details

Forgot password? Click here to reset