The complexity of mean payoff games using universal graphs

12/17/2018
by   Nathanaël Fijalkow, et al.
0

We study the computational complexity of solving mean payoff games. This class of games can be seen as an extension of parity games, and they have similar complexity status: in both cases solving them is in NP∩coNP and not known to be in P. In a breakthrough result Calude, Jain, Khoussainov, Li, and Stephan constructed in 2017 a quasipolynomial time algorithm for solving parity games, which was quickly followed by two other algorithms with the same complexity. It has recently been shown that the notion of universal graphs captures the combinatorial structure behind all three algorithms and gives a unified presentation together with the best complexity to date. In this paper we investigate how these techniques can be extended, and more specifically we give upper and lower bounds on the complexity of algorithms using universal graphs for solving mean payoff games. We construct two new algorithms each focussing on one of the two following parameters: the largest weight N (in absolute value) appearing in the graph and the number k of weights. Our first algorithm improves the best known complexity by reducing the dependence on N from N to N^1 - 1/n, where n is the number of vertices in the graph. Our second algorithm runs in polynomial time for a fixed number k of weights, more specifically in O(m n^k), where m is the number of edges in the graph. We complement our upper bounds by providing in both cases almost matching lower bounds, showing the limitations of the approach. We show that using universal graphs we cannot hope to improve on the N^1 - 1/n dependence in N nor break the O(n^Ω(k)) barrier. In particular, universal graphs do not yield a quasipolynomial algorithm for solving mean payoff games.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

04/12/2021

The Theory of Universal Graphs for Infinite Duration Games

We introduce the notion of universal graphs as a tool for constructing a...
10/09/2021

The GKK Algorithm is the Fastest over Simple Mean-Payoff Games

We study the algorithm of Gurvich, Khachyian and Karzanov (GKK algorithm...
10/11/2018

Parity games and universal graphs

This paper is a contribution to the study of parity games and the recent...
07/27/2018

Universal trees grow inside separating automata: Quasi-polynomial lower bounds for parity games

Several distinct techniques have been proposed to design quasi-polynomia...
09/17/2021

New Algorithms for Combinations of Objectives using Separating Automata

The notion of separating automata was introduced by Bojanczyk and Czerwi...
01/13/2020

A Universal Attractor Decomposition Algorithm for Parity Games

An attractor decomposition meta-algorithm for solving parity games is gi...
10/03/2021

A Recursive Algorithm for Solving Simple Stochastic Games

We present two recursive strategy improvement algorithms for solving sim...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

A mean payoff game is played over a finite graph whose edges are labelled by integer weights. The interaction of the two players, called Eve and Adam, describe a path in the graph. The goal of Eve is to ensure that the average limit of the weights is non-negative.

Figure 1: A mean payoff game (the region dotted in red is the set of vertices from which Eve has a winning strategy, and blue for Adam).

Early results

The model of mean payoff games was introduced independently by Ehrenfeucht and Mycielski [EM79] and by Gurvich, Karzanov, and Khachiyan [GKK88]. A fundamental property proved in both papers is that such games are positionally determined, meaning that a winning strategy does not need any memory at all and depends only on the current vertex. This holds for both players and implies the intriguing complexity status of solving mean payoff games: the decision problem is in NP and in coNP

, but not known to be solvable in polynomial time. Very few such problems have been known, two prime examples being linear programming and primality testing, both eventually shown to be solvable in polynomial time. Note that a problem in

is unlikely to be NP-complete, since this would imply that . Hence such a problem is either solvable in polynomial time or an interesting piece in the landscape of computational complexity.

This is one of the reasons that makes the study of mean payoff games exciting. In addition to game theory, the study of mean payoff games is motivated by verification and synthesis of programs, as well as for their intricate connections to optimisation and linear programming.

Towards a polynomial time algorithm

The seminal paper of Zwick and Paterson [ZP96] relates mean payoff games to discounted payoff games and simple stochastic games, and most relevant to our work, constructs a pseudopolynomial algorithm for solving mean payoff games. An algorithm is said to be pseudopolynomial if it is polynomial when the weights are given in unary. More specifically, the algorithm constructed by Zwick and Paterson depends linearly in the largest weight appearing in the game. If the weights are given in unary, is indeed polynomial in the representation.

The question whether there exists a polynomial time algorithm for mean payoff games with the usual representation of weights, meaning in binary, is open.

Recent contributions

The algorithm constructed by Zwick and Paterson runs in time , where is the number of vertices, the number of edges, and the largest weight in the graph. This has been improved by Brim, Chaloupka, Doyen, Gentilini, and Raskin [BCD11], yielding an algorithm of complexity .

The model of mean payoff games is also studied by the optimisation community. Smale asked, as the 9th item in his list of problems for the 21st century, whether linear programming can be solved in strongly polynomial time [Sma98]. A recent result has connected this problem to mean payoff games: Allamigeon, Benchimol, Gaubert, and Joswig [ABGJ14] have shown that a strongly polynomial time semi-algebraic pivoting rule in linear programming would solve mean payoff games in strongly polynomial time.

Parity games

It is most instructive in this context to think of parity games as a subclass of mean payoff games. More specifically, they are mean payoff games whose weights are of the form for , and is the number of priorities. The breakthrough result of Calude, Jain, Khoussainov, Li, and Stephan [CJK17] constructed a quasipolynomial time algorithm for solving parity games. Following decades of exponential and subexponential algorithms, this very surprising result triggered further research: soon after two different quasipolynomial time algorithms were constructed, by Jurdziński and Lazić [JL17], and by Lehtinen [Leh18]. They report almost the same complexity, which is roughly .

Czerwiński, Daviaud, Fijalkow, Jurdziński, Lazić, and Parys [CDF18] contributed to our understanding of these three algorithms by giving

  • a framework defining a family of discrete algorithms for solving parity games in which all three quasipolynomial algorithms fit,

  • a combinatorial notion, namely universal trees, underlying all algorithms in the framework,

  • a quasipolynomial lower bound on the complexity of algorithms within this framework.

This paper provides a similar story for mean payoff games.

Universal graphs

Universal trees very specifically and neatly describe the hierarchical structure of parity games, but they do not extend to other games such as mean payoff games. The notion of universal graphs was developed by Colcombet and Fijalkow [CF18] to fill in this gap. Universal graphs are a generic combinatorial notion which can be instantiated to many different games, as soon as they are positionally determined for Eve.

It was shown that when instantiated to parity games, saturated universal graphs indeed are universal trees. Since all quasipolynomial time algorithms for parity games can be presented using universal graphs, this motivates the study of algorithms for solving mean payoff games using universal graphs. Lower bounds for such algorithms provide evidence of the difficulty of extending the recent ideas for solving parity games to mean payoff games.

The notion of universal graphs has been extensively studied in an unrelated context of labelling schemes, that seek to assign a short bitstring (called a label) to every node of a graph, so that a query concerning two nodes can be answered by looking at their corresponding labels alone. As a prime example, labelling nodes of an undirected graph for adjacency queries is known to be equivalent to constructing a so-called induced-universal graph [KNR92]. Even though for some queries approaches based on an appropriately chosen notion of universal graphs are known to be suboptimal [FGNW17], we also have examples in which they allow for a significantly simpler and more efficient solution [GKŁ18].

Contributions

This paper is concerned with the computational complexity of solving mean payoff games. The best algorithm to date has complexity , where is the number of vertices, the number of edges, and the largest weight. A fourth parameter is relevant: , the number of distinct weights.

This paper gives upper and lower bounds on the complexity of algorithms solving mean payoff games using universal graphs, and in particular yields an improvement over the previously known algorithms.

We start by defining universal graphs and showing how they can be used to reduce mean payoff games to safety games. In other words, we show that to construct algorithms for solving mean payoff games it is enough to construct small universal graphs. We give a first construction yielding an algorithm of complexity , matching the best algorithm to date.

The appeal and beauty of the universal graph technology is that from this point onwards, we do not talk about games anymore. Indeed, the rest of the paper proves upper and lower bounds on the size of universal graphs. Our main results are the following.

Universal graphs parametrised by the largest weight

(Section 4)

  • There exists a -universal graph of size , which is bounded by .

  • All -universal graphs have size at least .

Since is polynomial in the size of the input, we can say that is “small”, while is exponential in the size of the input when weights are given in binary, hence “large”. The multiplicative gap between and upper and lower bound is bounded by , hence small.

As a consequence of the upper bound we obtain an algorithm solving mean payoff games with a small improvement over the algorithm: the dependence in goes from to . The lower bound shows that this dependence cannot be improved.

Universal graphs parametrised by the number of weights

(Section 5)

  • For all of size , there exists a -universal graph of size .

  • For all , for large enough, there exists of size such that all -universal graphs have size at least .

As a consequence, we obtain an algorithm solving mean payoff games of complexity . The lower bound shows that algorithms using universal graphs cannot break the barrier, and in particular do not have quasipolynomial complexity.

2 Definitions

We write for the intervall , and use parenthesis to exclude extremal values, so is .

Graphs

We consider graphs labelled by integers: means that there is an edge from to labelled by . A graph is given by a set of vertices, a set of edges, and an initial vertex . We always assume that all vertices are reachable from . We let denote the number of vertices and the number of edges. The size of a graph is its number of vertices and denoted by .

A path is a (finite or infinite) sequence of consecutive triples in . Consecutive means that the third component of a triple in the sequence matches the first component of the next triple. In the case of a finite path we write for the last vertex in . We write and let denote the prefix of of length , meaning .

Games

A mean payoff game is given by a graph together with two sets and such that . We often let denote a mean payoff game. The set is the set of vertices controlled by Eve (represented by circles in Fig 1) and the set of vertices controlled by Adam (represented by squares in Fig 1). Let be a finite subset of the integers, we speak of an -mean payoff game if it has size and all the weights labelling edges are in .

The interaction between the two players goes as follows. A token is initially placed on the initial vertex , and the player who controls this vertex pushes the token along an edge, reaching a new vertex; the player who controls this new vertex takes over, and this interaction goes on potentially forever, describing a path. In the context of mean payoff games, we say that an infinite path satisfies mean payoff, or equivalently is winning, if

A strategy is a map . Note that we always take the point of view of Eve, so a strategy implicitely means a strategy of Eve, and winning means winning for Eve. We say that a path is consistent with the strategy if for all , if , then

A strategy is winning from if all paths starting in are infinite and winning. Solving a mean payoff game is the following decision problem:

INPUT: a mean payoff game
OUTPUT: YES if Eve has a winning strategy from , NO otherwise.

Of special importance are positional strategies, which are given by

such strategies make decisions only considering the current vertex. A positional strategy induces a strategy by , where by convention the last vertex of the empty path is .

Theorem 1 ([Em79]).

For all mean payoff games, if Eve has a winning strategy, then she has a positional winning strategy.

We say that a graph satisfies mean payoff if all infinite paths from are winning and there are no dead ends. This can be easily characterized using cycles: a cycle is a path of finite length such that . A cycle is negative if

Fact 1.

A graph satisfies mean payoff if and only if it does not contain any negative cycle.

Given a mean payoff game and a positional strategy , we let denote the graph obtained by restricting to the moves prescribed by . Formally, the set of vertices and edges are

Fact 2.

Let be a mean payoff game and a positional strategy. Then the graph satisfies mean payoff if and only if is winning.

A safety game is played over an unlabelled graph, meaning the set of edges is . As for a mean payoff game, it is given by a graph together with two sets and such that . The word safety refers to the winning condition, which is in some sense the simplest possible: Eve wins if she can play forever, meaning any infinite play is winning. Eve loses if a play cannot be prolonged.

The notions of paths and strategies are inherited from mean payoff games, and the positional determinacy stated in Theorem 1 also holds for safety games. The following lemma is folklore.

Lemma 1.

There exists an algorithm running in time determining for each vertex of a safety game whether Eve has a winning strategy from .

3 Universal graphs

For two graphs , a homomorphism maps the vertices of to the vertices of such that

Homomorphisms do not have any requirement about the initial vertices.

Definition 1.

A graph is -universal if it does not contain any negative cycle and any -graph without negative cycles can be mapped homomorphically into it.

We show later in this section how universal graphs can be used for solving mean payoff games, yielding algorithms whose complexity is proportional to the size of the universal graph. It is not even clear at this point that there exists a universal graph, let alone a small one. Indeed, the definition creates a tension between “not containing any negative cycle”, suggesting that the graph is small, and “contains any -graph without negative cycles”, suggesting that the graph is large. However, one can see that an -universal graph exists by taking the disjoint union of all -graphs without negative cycles. Indeed, up to renaming of vertices there are finitely many such graphs, so this yields a very large but finite universal graph. We will show that there are much smaller ones.

A linear graph is given by a finite subset of the integers. The set of vertices is and for any and there is an edge from to labelled if . Observe that linear graphs do not contain negative cycles.

Given a graph , the distance from a vertex to a vertex is the smallest sum of the weights along a path from and .

Lemma 2.

Let be a graph without negative cycles. We let define the linear graph

Then homomorphically maps into .

Proof.

Note that distances are well defined precisely because does not have negative cycles. We define mapping a vertex of to . Since we assume that all vertices are reachable from , this is well defined. We claim that is a homomorphism from to , which follows from the triangle inequality: if , then

or equivalently . ∎

Corollary 1.

The linear graph is -universal.

Proof.

We observe that the linear graph constructed in Lemma 2 satisfies , which implies that is -universal. ∎

Lemma 2 gives more than a universal graph: it shows that for every universal graph, there is a linear universal graph of the same size. Indeed, let be a -universal graph, then maps homomorphically into the linear graph , and the size of (meaning the number of vertices) is no larger than the size of . This implies that is -universal, since any -graph without negative cycles homomorphically maps into , which composed with the homomorphism into yields a homomorphism from to .

We state here a simple fact that we will use later on about homomorphisms from linear graphs.

Fact 3.

Let be linear graphs, a homomorphism, and , then .

The inequality is by definition of the homomorphism using the edge , and the converse inequality is obtained by considering the edge .

Solving mean payoff games using universal graphs

Let be a -universal graph. Thanks to the remark above we can assume without loss of generality that is a linear graph. In particular there is a natural order on vertices (which are integers). We use this order to give two definitions:

  • For and , we let be the largest such that , or equivalently with the set of edges of .

  • We let be the largest vertex in .

We reduce a -mean payoff game to a safety game . Intuitively, we replace the mean payoff condition by a simpler one, which is to stay forever in . Formally, we let denote the underlying graph of . In the set of vertices and edges are

The initial vertex is .

Lemma 3.

Let be a -mean payoff game and a -universal graph. Then Eve has a winning strategy in the mean payoff game if and only if she has a winning strategy in the safety game .

Proof.

Let us assume that Eve has a winning strategy in the mean payoff game , which can be chosen to be positional thanks to Theorem 1. Since is winning the graph satisfies mean payoff thanks to Fact 2. Since is -universal there exists a homomorphism from to .

We define the positional strategy in the safety game by

To see that is a winning strategy, we observe that for any path consistent with whose last vertex is , we have . This is initially true by definition of . We show that it is preserved: let be the last edge in , by definition of it is an edge in since it was either picked by Adam or by Eve following .

Conversely, a winning strategy in the safety game from can be seen as a strategy in the mean payoff game . Any path consistent with is a path in , and since satisfies mean payoff this implies that all paths consistent with satisfy mean payoff, in other words is winning in . ∎

This lemma yields an algorithm for solving mean payoff games whose complexity is proportional to the size of .

Theorem 2.

Given a -universal graph , we can construct an algorithm solving -mean payoff games of complexity

The algorithm we obtain using the simple construction above combined with Theorem 2 matches the best complexity so far.

Corollary 2.

There exists an algorithm for solving mean payoff games of complexity .

Constructing universal graphs

Let us recall the results for parity games, which form the class of mean payoff games whose set of weights is

We can now formulate in a more technical way the results for parity games.

Theorem 3 ([Cdf18, Cf18]).

For all , let .

  • There exists a -universal graph of size .

  • All -universal graphs have size at least .

There are three constructions for the upper bound, one for each quasipolynomial time algorithm: the algorithm constructed by Calude, Jain, Khoussainov, Li, and Stephan [CJK17], the algorithm constructed by Jurdziński and Lazić [JL17], and the algorithm constructed by Lehtinen [Leh18]. We note that the complexity reported in their analysis is not made worse by rephrasing the algorithms using universal graphs, and even in some cases very slightly improved.

The technical core of this paper is to extend this study to arbitrary sets of weights , inducing algorithms for solving mean payoff games. We consider two parameters on : the largest weight is absolute value, in other words the case where , and the number of weights, i.e. the size of .

4 Parametrised by the largest weight

In this section we focus on the largest weight of in absolute value as parameter, so we fix .

We already explained how to construct a -universal graph of size , yielding an algorithm with the best known complexity. In this section we show the following improved results.

Theorem 4.

For all ,

  • There exists a -universal graph of size , which is upper bounded by .

  • All -universal graphs have size at least .

Corollary 3.

There exists an algorithm for solving mean payoff games of complexity

Upper bound

Proposition 1.

There exists a -universal graph of size .

Before giving the formal construction, we give some intuitions. The simple construction in Lemma 2 shows that the linear graph is -universal. Some extra care would actually yield that the linear graph is also -universal, a marginal gain. In this construction the initial vertex is always mapped to by the homomorphism. The idea of the improved upper bound is that by lifting this restriction we recover some slack, and by translating the position of the initial vertex in the linear graph we can remove from some vertices: the added slack allows to avoid them in any homomorphism. The size of the removed set is .

Proof.

Let . We write all integers in in base , hence using digits: represents

We let be the set of integers in which have at least one zero digit in this decomposition. We argue that is a -universal graph.

Let be a graph of size without negative cycles. Thanks to Lemma 2 the graph homomorphically maps into a linear graph . We let

We let for . We argue that , which follows from the observation that for any , either or there exists such that and . Indeed, recall that . We proceed by induction on the length of the shortest path witnessing . For , either or with , so . In case we rely on the inductive hypothesis to conclude.

We construct a homomorphism from to , which composed with the homomorphism from to yields a homomorphism from to . We note that is actually fully determined by : indeed, let , since there is an edge in we must have , thanks to Fact 3. It follows that .

We let , i.e. is the decomposition of . We explain how to choose .

  • We first choose , which ensures that , implying .

  • For we have and since and we have . Setting ensures that , so .

  • More generally using the fact that and we have so that setting ensures , implying .

This completes the construction of . The size of is . ∎

Lower bound

Proposition 2.

Any -universal graph has size at least .

Proof.

Let be a -universal graph. Thanks to Lemma 2 we can assume that is linear. We construct an injective function

For , we consider the linear graph

which has size , hence homomorphically maps into : let . We let .

To see that is injective, we note that since is a homomorphism and thanks to Fact 3 we have

Since is injective this implies that

i.e. . ∎

5 Parametrised by the number of weights

In this section we focus on the size of as a parameter.

Theorem 5.

For all ,

  • For all , for all of size , there exists a -universal graph of size .

  • For large enough, there exists of size such that all -universal graphs have size at least .

Corollary 4.

There exists an algorithm for solving mean payoff games with weights of complexity .

Upper bound

Define to be the number of different sums of terms of .

Proposition 3.

There exists a -universal graph of size .

Proof.

We simply observe that the linear graph constructed in Lemma 2 is included in the set of different sums of terms of . ∎

It follows that there exists an algorithm for solving mean payoff games of complexity . In particular for this yields an algorithm in , which is polynomial for a constant .

Lower bound

We let

and

Note that has indeed size .

Proposition 4.

Let be a -universal graph. Then

Proof.

Let be a -universal graph. Thanks to Lemma 2 we can assume that is linear.

We construct a class of -graphs which are cycles of length , as follows. Let . The vertices are and there is an edge for , and an edge . To make the total weight in the cycle equal to , we assume that each appears exactly many times in .

We will not use all graphs described above, but only a subset which we describe in the following way. We let be the set of sequences of integers in such that , where we use the notation for an element . A tuple of sequences in induces a -graph . Let , the induced graph is partitioned into parts. In the th part the weight is used exactly many times.

Figure 2: Construction of the graph from the sequences .

In the drawing the parts are represented by boxes and numbered from to with a number in parenthesis. To induce the box in this drawing we have , and .

The vertex in marking the end of the first box is , and the vertex marking the end of the th box is

We note that writing the number in base we recover since this is a number in , hence in particular in . Hence doing this for fully determines the sequences . We also let .

The constraint on the sums of sequences on ensures that indeed the graph is a cycle of total weight . Hence there exists a homomorphism . We construct an injective function

To see that is injective, we note that since is a homomorphism we have

As explained above the numbers fully determine the sequences .

Since is injective this implies that . The size of is

which implies (for constant)

Conclusions

In this paper we have shown how to extend to mean payoff games the ideas developed for constructing quasipolynomial algorithms for parity games. This relies on the combinatorial notion of universal graphs. We give upper bounds, yielding two new algorithms with the best complexity to date, and lower bounds, showing the limitations of this approach. In particular, algorithms based on universal graphs cannot solve mean payoff games in quasipolynomial time.

More precisely, our lower bounds show that for pathological sets of weights universal graphs are very large. A more positive note is to consider , the set of weights corresponding to parity games: in this case we know that there exist -universal graphs of quasipolynomial size (specifically ). This motivates a deeper understanding of the size of -universal graphs: for which sets of weights do there exist small universal graphs?

References

  • [ABGJ14] Xavier Allamigeon, Pascal Benchimol, Stéphane Gaubert, and Michael Joswig. Combinatorial simplex algorithms can solve mean payoff games. SIAM Journal on Optimization, 24(4):2096–2117, 2014.
  • [BCD11] Lubos Brim, Jakub Chaloupka, Laurent Doyen, Raffaella Gentilini, and Jean-François Raskin. Faster algorithms for mean-payoff games. Formal Methods in System Design, 38(2):97–118, 2011.
  • [CDF18] Wojciech Czerwiński, Laure Daviaud, Nathanaël Fijalkow, Marcin Jurdziński, Ranko Lazić, and Paweł Parys. Universal trees grow inside separating automata: Quasi-polynomial lower bounds for parity games. CoRR, abs/1807.10546, 2018.
  • [CF18] Thomas Colcombet and Nathanaël Fijalkow. Parity games and universal graphs. CoRR, abs/1810.05106, 2018.
  • [CJK17] Cristian S. Calude, Sanjay Jain, Bakhadyr Khoussainov, Wei Li, and Frank Stephan. Deciding parity games in quasipolynomial time. In STOC, pages 252–263, 2017.
  • [EM79] Andrzej Ehrenfeucht and Jan Mycielski. Positional strategies for mean payoff games. International Journal of Game Theory, 109(8):109–113, 1979.
  • [FGNW17] Ofer Freedman, Paweł Gawrychowski, Patrick K. Nicholson, and Oren Weimann. Optimal distance labeling schemes for trees. In PODC, pages 185–194. ACM, 2017.
  • [GKK88] Vladimir A. Gurvich, Aleksander V. Karzanov, and Leonid G. Khachiyan. Cyclic games and an algorithm to find minimax cycle means in directed graphs. USSR Computational Mathematics and Mathematical Physics, 28:85–91, 1988.
  • [GKŁ18] Paweł Gawrychowski, Fabian Kuhn, Jakub Łopuszański, Konstantinos Panagiotou, and Pascal Su. Labeling schemes for nearest common ancestors through minor-universal trees. In SODA, pages 2604–2619. SIAM, 2018.
  • [JL17] Marcin Jurdziński and Ranko Lazić. Succinct progress measures for solving parity games. In LICS, pages 1–9, 2017.
  • [KNR92] Sampath Kannan, Moni Naor, and Steven Rudich. Implicit representation of graphs. SIAM Journal on Discrete Mathematics, 5(4):596–603, 1992.
  • [Leh18] Karoliina Lehtinen. A modal- perspective on solving parity games in quasi-polynomial time. In LICS, pages 639–648, 2018.
  • [Sma98] Steve Smale. Mathematical problems for the next century. The Mathematical Intelligencer, 20(2):7–15, 1998.
  • [ZP96] Uri Zwick and Mike Paterson. The complexity of mean payoff games on graphs. Theoretical Computer Science, 158(1&2):343–359, 1996.