POMDPs under Probabilistic Semantics

08/09/2014
by   Krishnendu Chatterjee, et al.
0

We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the rewards. We consider two types of path constraints: (i) quantitative constraint defines the set of paths where the payoff is at least a given threshold lambda_1 in (0,1]; and (ii) qualitative constraint which is a special case of quantitative constraint with lambda_1=1. We consider the computation of the almost-sure winning set, where the controller needs to ensure that the path constraint is satisfied with probability 1. Our main results for qualitative path constraint are as follows: (i) the problem of deciding the existence of a finite-memory controller is EXPTIME-complete; and (ii) the problem of deciding the existence of an infinite-memory controller is undecidable. For quantitative path constraint we show that the problem of deciding the existence of a finite-memory controller is undecidable.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/30/2019

The Complexity of POMDPs with Long-run Average Objectives

We study the problem of approximation of optimal values in partially-obs...
research
04/27/2022

Bounds for Synchronizing Markov Decision Processes

We consider Markov decision processes with synchronizing objectives, whi...
research
04/10/2018

Combinations of Qualitative Winning for Stochastic Parity Games

We study Markov decision processes and turn-based stochastic games with ...
research
12/21/2018

Expected Window Mean-Payoff

In the window mean-payoff objective, given an infinite path, instead of ...
research
05/14/2020

Qualitative Controller Synthesis for Consumption Markov Decision Processes

Consumption Markov Decision Processes (CMDPs) are probabilistic decision...
research
05/31/2021

LTL-Constrained Steady-State Policy Synthesis

Decision-making policies for agents are often synthesized with the const...
research
01/25/2022

Probabilistic Total Store Ordering

We present Probabilistic Total Store Ordering (PTSO) – a probabilistic e...

Please sign up or login with your details

Forgot password? Click here to reset