Log In Sign Up

Strategy Complexity of Parity Objectives in Countable MDPs

by   Stefan Kiefer, et al.

We study countably infinite MDPs with parity objectives. Unlike in finite MDPs, optimal strategies need not exist, and may require infinite memory if they do. We provide a complete picture of the exact strategy complexity of ε-optimal strategies (and optimal strategies, where they exist) for all subclasses of parity objectives in the Mostowski hierarchy. Either MD-strategies, Markov strategies, or 1-bit Markov strategies are necessary and sufficient, depending on the number of colors, the branching degree of the MDP, and whether one considers ε-optimal or optimal strategies. In particular, 1-bit Markov strategies are necessary and sufficient for ε-optimal (resp. optimal) strategies for general parity objectives.


Transience in Countable MDPs

The Transience objective is not to visit any state infinitely often. Whi...

Combinations of Qualitative Winning for Stochastic Parity Games

We study Markov decision processes and turn-based stochastic games with ...

Aspects of Muchnik's paradox in restricted betting

Muchnik's paradox says that enumerable betting strategies are not always...

Strategy Complexity of Point Payoff, Mean Payoff and Total Payoff Objectives in Countable MDPs

We study countably infinite Markov decision processes (MDPs) with real-v...

Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints

We formalize the problem of maximizing the mean-payoff value with high p...

Mixing Probabilistic and non-Probabilistic Objectives in Markov Decision Processes

In this paper, we consider algorithms to decide the existence of strateg...

Threshold Constraints with Guarantees for Parity Objectives in Markov Decision Processes

The beyond worst-case synthesis problem was introduced recently by Bruyè...