The Complexity of POMDPs with Long-run Average Objectives

04/30/2019
by   Krishnendu Chatterjee, et al.
0

We study the problem of approximation of optimal values in partially-observable Markov decision processes (POMDPs) with long-run average objectives. POMDPs are a standard model for dynamic systems with probabilistic and nondeterministic behavior in uncertain environments. In long-run average objectives rewards are associated with every transition of the POMDP and the payoff is the long-run average of the rewards along the executions of the POMDP. We establish strategy complexity and computational complexity results. Our main result shows that finite-memory strategies suffice for approximation of optimal values, and the related decision problem is recursively enumerable complete.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/01/2021

Strategy Complexity of Mean Payoff, Total Payoff and Point Payoff Objectives in Countable MDPs

We study countably infinite Markov decision processes (MDPs) with real-v...
research
08/09/2014

POMDPs under Probabilistic Semantics

We consider partially observable Markov decision processes (POMDPs) with...
research
08/17/2021

On the equivalence of holding cost and response time for evaluating performance of queues

This self-contained discussion relates the long-run average holding cost...
research
03/10/2022

Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPs

We study countably infinite Markov decision processes (MDPs) with real-v...
research
10/26/2020

Multi-objective Optimization of Long-run Average and Total Rewards

This paper presents an efficient procedure for multi-objective model che...
research
07/11/2017

Synthesis of Optimal Resilient Control Strategies

Repair mechanisms are important within resilient systems to maintain the...
research
06/20/2017

Mean-Payoff Optimization in Continuous-Time Markov Chains with Parametric Alarms

Continuous-time Markov chains with alarms (ACTMCs) allow for alarm event...

Please sign up or login with your details

Forgot password? Click here to reset