Combinations of Qualitative Winning for Stochastic Parity Games

by   Krishnendu Chatterjee, et al.

We study Markov decision processes and turn-based stochastic games with parity conditions. There are three qualitative winning criteria, namely, sure winning, which requires all paths must satisfy the condition, almost-sure winning, which requires the condition is satisfied with probability 1, and limit-sure winning, which requires the condition is satisfied with probability arbitrarily close to 1. We study the combination of these criteria for parity conditions, e.g., there are two parity conditions one of which must be won surely, and the other almost-surely. The problem has been studied recently by Berthon et. al for MDPs with combination of sure and almost-sure winning, under infinite-memory strategies, and the problem has been established to be in NP ∩ coNP. Even in MDPs there is a difference between finite-memory and infinite-memory strategies. Our main results for combination of sure and almost-sure winning are as follows: (a) we show that for MDPs with finite-memory strategies the problem lie in NP ∩ coNP; (b) we show that for turn-based stochastic games the problem is coNP-complete, both for finite-memory and infinite-memory strategies; and (c) we present algorithmic results for the finite-memory case, both for MDPs and turn-based stochastic games, by reduction to non-stochastic parity games. In addition we show that all the above results also carry over to combination of sure and limit-sure winning, and results for all other combinations can be derived from existing results in the literature. Thus we present a complete picture for the study of combinations of qualitative winning criteria for parity conditions in MDPs and turn-based stochastic games.


page 1

page 2

page 3

page 4


Strategy Complexity of Parity Objectives in Countable MDPs

We study countably infinite MDPs with parity objectives. Unlike in finit...

Parity Games with Weights

Quantitative extensions of parity games have recently attracted signific...

Simple Stochastic Games with Almost-Sure Energy-Parity Objectives are in NP and coNP

We study stochastic games with energy-parity objectives, which combine q...

Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints

We formalize the problem of maximizing the mean-payoff value with high p...

POMDPs under Probabilistic Semantics

We consider partially observable Markov decision processes (POMDPs) with...

Extending finite-memory determinacy by Boolean combination of winning conditions

We study finite-memory (FM) determinacy in games on finite graphs, a cen...

Strategy Representation by Decision Trees with Linear Classifiers

Graph games and Markov decision processes (MDPs) are standard models in ...