DeepAI
Log In Sign Up

Strategy Complexity of Parity Objectives in Countable MDPs

07/07/2020
by   Stefan Kiefer, et al.
0

We study countably infinite MDPs with parity objectives. Unlike in finite MDPs, optimal strategies need not exist, and may require infinite memory if they do. We provide a complete picture of the exact strategy complexity of ε-optimal strategies (and optimal strategies, where they exist) for all subclasses of parity objectives in the Mostowski hierarchy. Either MD-strategies, Markov strategies, or 1-bit Markov strategies are necessary and sufficient, depending on the number of colors, the branching degree of the MDP, and whether one considers ε-optimal or optimal strategies. In particular, 1-bit Markov strategies are necessary and sufficient for ε-optimal (resp. optimal) strategies for general parity objectives.

READ FULL TEXT
12/26/2020

Transience in Countable MDPs

The Transience objective is not to visit any state infinitely often. Whi...
04/10/2018

Combinations of Qualitative Winning for Stochastic Parity Games

We study Markov decision processes and turn-based stochastic games with ...
01/14/2022

Aspects of Muchnik's paradox in restricted betting

Muchnik's paradox says that enumerable betting strategies are not always...
03/10/2022

Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPs

We study countably infinite Markov decision processes (MDPs) with real-v...
04/24/2018

Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints

We formalize the problem of maximizing the mean-payoff value with high p...
04/28/2020

Mixing Probabilistic and non-Probabilistic Objectives in Markov Decision Processes

In this paper, we consider algorithms to decide the existence of strateg...
02/17/2017

Threshold Constraints with Guarantees for Parity Objectives in Markov Decision Processes

The beyond worst-case synthesis problem was introduced recently by Bruyè...