Two-player zero-sum graph games with -regular objectives are the classical mathematical model to formalize the reactive synthesis problem [PnueliR89, Thomas95]. More recently, generalization from zero-sum to non zero-sum, and from two players to players have been considered in the literature, see e.g. [berwanger07, BrenguierCHPRRS16, BRS-concur15, BRS14, BMR14, KHJ06, FismanKL10, KupfermanPV14, Ummels06] and the surveys [Bruyere17, GU08]. Those extensions are motivated by two main limitations of the classical setting. First, zero-sum games assume a fully antagonistic environment while this is often not the case in practice: the environment usually has its own goal. While the fully antagonistic assumption is simple and sound (a winning strategy against an antagonistic environment is winning against any environment that pursues its own objective), it may fail to find a winning strategy even if solutions exist when the objective of the environment is accounted. Second, modern reactive systems are often composed of several modules, and each module has its own specification and should be considered as a player on its own right. This is why we need to consider -player graph games.
For -player graph games, solution concepts like Nash equilibria (NEs) [nash50] are natural to consider. A strategy profile is an NE if no player has an incentive to deviate unilaterally from his strategy, i.e. no player can strictly improve on the outcome of the strategy profile by changing his strategy only. In the context of sequential games (such as games played on graphs), NEs allow for non-credible threats that rational players should not carry out. To avoid non-credible threats, refinements such as subgame perfect equilibria (SPEs) [osbornebook] have been advocated. A strategy profile is an SPE if it is an NE in all the subgames of the original game. So players need to play rationally in all subgames, and this ensures that non-credible threats cannot exist. For applications of this concept to -player graph games, we refer the reader to [BrihayeBDG12, KHJ06, Ummels06].
In [BrihayeBMR15], the notion of weak subgame perfect equilibrium (weak SPE) is introduced, and it is shown how it can be used to study the existence SPEs (possibly with contraints) in quantitative reachability games. While an SPE must be resistant to any unilateral deviation of one player, a weak SPE must be resistant to deviations restricted to deviating strategies that differ from the original one on a finite number of histories only. In [Bruyere0PR17] the authors study general conditions on the structure of the game graph and on the preference relations of the players that guarantee the existence of a weak SPE for quantitative games. Weak SPEs retain most of the important properties of SPEs and they coincide with them when the payoff function of each player is continuous (see e.g. [fudenberg1991game]). Weak SPEs are also easier to characterize and to manipulate algorithmically. We refer the interested reader to [BrihayeBMR15, Bruyere0PR17] for further justifications of their interest, as well as for related work on NEs and SPEs.
In this paper, we concentrate on graph games with -regular Boolean objectives. While SPEs, and thus weak SPEs, are always guaranteed to exist in such games, we here study the computational complexity of the constrained existence problem for weak SPEs, i.e. equilibria in which some designated players have to win and some other ones have to loose. More precisely, our main results are as follows:
We study the constrained existence problem for games with Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, and Streett objectives. We provide a complete characterization of the computational complexity of this problem for all the classes of objectives with one exception: Büchi objectives. The problem is P-complete for Explicit Muller objectives, it is NP-complete for Co-Büchi, Parity, Muller, Rabin, and Streett objectives, and it is PSPACE-complete for Reachability and Safety objectives. In case of Büchi objectives, we show membership to NP but we fail to prove hardness.
Our complexity results rely on the identification of a symbolic witness for the constrained existence of a weak SPE, the size of which allows us to prove NP/PSPACE-membership. As the constrained existence problem is PSPACE-complete for Reachability and Safety objectives, symbolic witnesses as compact as those for the other objectives cannot exist unless NP PSPACE. The identification of symbolic witnesses is obtained thanks to a fixpoint algorithm that computes the set of all possible payoff profiles underlying weak SPEs.
When the number of players is fixed, we show that the constrained existence problem can be solved in polynomial time for all -regular objectives. We also prove that it is fixed parameter tractable where the parameter is the number of players, for Reachability, Safety, Büchi, Co-Büchi, and Parity objectives. For Rabin, Streett, and Muller objectives, we still establish fixed parameter tractability but we need to consider some additional parameters depending on the objectives. These tractability results are obtained by a fine analysis of the complexity of the fixpoint algorithm mentioned previously.
Related work and additional contributions
In [GU08, Ummels06], a tree automata-based algorithm is given to decide the constrained existence problem for SPEs on graph games with -regular objectives defined by parity conditions. A complexity gap is left open: this algorithm executes in EXPTIME and NP-hardness of the decision problem is proved. In this paper, we focus on weak SPEs for which we provide precise complexity results for the constrained existence problem. We also observe that our results on Reachabilty and Safety objectives transfer from weak SPEs to SPEs: the constrained existence problem for SPEs is PSPACE-complete for those objectives. Quantitative Reachability objectives are investigated in [BrihayeBMR15] where it is proved that the constrained existence problem for weak SPEs and SPEs is decidable, but its exact complexity is left open.
In [BrihayeBMR15, Bruyere0PR17, FleschKMSSV10], the existence of (weak) SPEs in graph games is established using a construction based on a fixpoint. Our fixpoint algorithm is mainly inspired by the fixpoint technique of [Bruyere0PR17]. However, we provide complexity results based on this fixpoint while transfinite induction is used in [Bruyere0PR17]. Furthermore, we have modified the technique of [Bruyere0PR17] in a way to get a fixpoint that contains exactly all the possible payoff profiles of weak SPEs. This is necessary to get a decision algorithm for the constrained existence problem.
Profiles of strategies with finite-memory are more appealing from a practical point of view. It is shown in [Ummels06] that when there exists an SPE in a graph game with -regular objectives, then there exists one that uses finite-memory strategies and has the same payoff profile. Thanks to the symbolic witnesses, we have refined those results for weak SPEs.
Structure of the paper
In Section 2, we recall the notions of -player graph games and of (weak) SPE, and we state the studied constrained existence problem. In Section 3, we provide a fixpoint algorithm that computes all the possible payoff profiles for weak SPEs on a given graph game. From this fixpoint, we derive symbolic witnesses of weak SPEs. In Section 4, we study the complexity classes of the constrained existence problem for all objectives. We also prove the fixed parameter tractability of the constrained existence problem and we show that it is in polynomial time when the number of players is fixed. In Section 5, we give a conclusion and propose future work.
In this section, we introduce multiplayer graph games in which each player aims to achieve his Boolean objective. We focus on classical -regular objectives, like Reachability, Büchi, aso. We recall two classical concepts of equilibria: Nash equilibrium and subgame perfect equilibrium (see [GU08]). We also recall weak variants of these equilibria as proposed in [BrihayeBMR15, Bruyere0PR17]. We conclude the section by the constrained existence problem that is studied in this paper.
Multiplayer Boolean games
A multiplayer Boolean game is a tuple where is a finite set of players, is a finite directed graph and for all there exists such that , is a partition of between the players, and is a tuple of functions that assigns a Boolean value to each infinite path of for player .
A play in is an infinite sequence of vertices such that for all , . A history is a finite sequence () defined similarly. We denote the set of plays by and the set of histories by . Moreover, the set is the set of histories such that the last vertex is a vertex of player , i.e. . The length of is the number of its edges. A play is called a lasso if it is of the form with . Notice that is not necessary a simple cycle. The length of a lasso is the length of . For all , we denote by the first vertex of . We use notation when a history is prefix of a play (or a history) . Given a play , the set is the set of vertices visited by , and is the set of vertices infinitely often visited by . Given a vertex , is the set of successors of , and is the set of vertices reachable from in .
When an initial vertex is fixed, we call an initialized game. A play (resp. a history) of is a play (resp. a history) of starting in . The set of such plays (resp. histories) is denoted by (resp. ). We also use notation when these histories end in a vertex .
The goal of each player is to achieve his objective, i.e., to maximize his gain.
Definition 1 (Objective).
For each player , let be his objective. In the setting of multiplayer Boolean game, the gain function is defined such that (resp. ) if and only if (resp. ).
An objective (or the related gain function ) is prefix-independent if for all and , we have if and only if . In this paper, we focus on classical -regular objectives: Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, and Streett and we suppose that each player has the same type of objective. For instance, we say that is a Boolean game with Büchi objectives to express that all players have a Büchi objective.
Definition 2 (Classical -regular objective).
The set is a Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, or Streett objective for player if and only if is composed of the plays satisfying:
Reachability: given , ;
Safety: given , ;
Büchi: given , ;
Co-Büchi: given , ;
Parity: given a coloring function , ;
Explicit Muller: given , ;
Muller: given a coloring function , and , ;
Rabin: given a family of pair of sets ,
there exists such that and ;
Streett: given a family of pair of sets ,
for all , or .
All these objectives are prefix-independent except Reachability and Safety objectives.
A strategy of a player is a function . This function assigns to each history with , a vertex such that . In an initialized game , needs only to be defined for histories starting in . A play is consistent with if for all we have that . A strategy is positional if it only depends on the last vertex of the history, i.e., for all . It is finite-memory if it can be encoded by a deterministic Moore machine where is a finite set of states (the memory of the strategy), is the initial memory state, is the update function, and is the next-action function. The Moore machine defines a strategy such that for all histories , where extends to histories as expected. The size of the strategy is the size of its machine . Note that is positional when .
A strategy profile is a tuple of strategies, one for each player. It is called positional (resp. finite-memory) if for all , is positional (resp. finite-memory). Given an initialized game and a strategy profile , there exists an unique play from consistent with each strategy . We call this play the outcome of and it is denoted by . Let , we say that is a strategy profile with payoff or that has payoff if for all .
In the multiplayer game setting, the solution concepts usually studied are equilibria (see [GU08]). We here recall the concepts of Nash equilibrium and subgame perfect equilibrium, as well as some variants. We begin with the notion of deviating strategy.
Let be a strategy profile in an initialized Boolean game . Given , a strategy is a deviating strategy of player , and denotes the strategy profile where replaces . Such a strategy is a profitable deviation for player if . We say that is finitely deviating from if and only differ on a finite number of histories, and that is one-shot deviating from if and only differ on [BrihayeBMR15, Bruyere0PR17].
The notion of Nash equilibrium (NE) is classical: a strategy profile in an initialized game is a Nash equilibrium if no player has an incentive to deviate unilaterally from his strategy since he has no profitable deviation, i.e., for each and each deviating strategy of player from , the following inequality holds: . In this paper we focus on two variants of NE: weak/very weak NE [BrihayeBMR15, Bruyere0PR17].
Definition 3 (Weak/very weak Nash equilibrium).
A strategy profile is a weak NE (resp. very weak NE) in if, for each player , for each finitely deviating (resp. one-shot) strategy of player from , we have .
Figure 1 illustrates an initialized Boolean game with Büchi objectives in which there exists a weak NE that is not an NE. In this game, player 1 (resp. player 2) owns round (resp. square) vertices and wants to visits (resp. or ) infinitely often. The positional strategy profile is depicted by dashed arrows, its outcome is equal to , and has payoff . Notice that player 1 has an incentive to deviate from his strategy with a strategy that goes to for all histories ending in . This is indeed a profitable deviation for him since . So, is not an NE. Nevertheless, it is a weak NE because is the only profitable deviation and it is not finitely deviating (it differs from on all histories of the form for ).
When considering games played on graphs, a well-known refinement of NE is the concept of subgame perfect equilibrium (SPE) which a strategy profile being an NE in each subgame. Variants of weak/very weak SPE can also be studied as done with NEs. Formally, given an initialized Boolean game and a history , the initialized game is called a subgame222Notice that is subgame of itself. of such that and for all and . Moreover if is a strategy for player in , then denotes the strategy in such that for all histories , . Similarly, from a strategy profile in , we derive the strategy profile in . The play is called a subgame outcome of .
Definition 4 (Subgame perfect equilibrium and weak/very weak subgame perfect equilibrium).
A strategy profile is a (resp. weak, very weak) subgame perfect equilibrium in if for all , is a (resp. weak, very weak) NE in .
When one needs to show that a strategy profile is a weak SPE, the next proposition is very useful because it states that it is enough to consider one-shot deviating strategies.
Proposition 1 ([BrihayeBMR15]).
A strategy profile is a weak SPE if and only if is a very weak SPE.
In Example 1 is given a weak NE in the game depicted in Figure 1. This strategy profile is also a very weak SPE (and thus a weak SPE by Proposition 1). For instance, in the subgame with and , the only one-shot deviating strategy is such that coincides with except that . This is not a profitable deviation for player in . Notice that is not an SPE since it is not an NE as explained in Example 1.
It is proved in [Bruyere0PR17] that there always exists a weak SPE in Boolean games. In this paper, we are interested in solving the following constraint problem:
Definition 5 (Constraint problem).
Given an initialized Boolean game and thresholds , decide whether there exists a weak SPE in with payoff such that .333The order is the componentwise order, that is, , for all .
In the next sections, we solve the constraint problem for the classical -regular objectives. The complexity classes that we obtain are shown in Table 1; they are detailed in Section 4. The case of Büchi objectives remains open, since we only propose a non-deterministic algorithm in polynomial time but no matching lower bound. In Section 4, we also prove that the constraint problem for weak SPEs is fixed parameter tractable and becomes polynomial when the number of players is fixed. All these results are based on a characterization of the set of possible payoffs of a weak SPE, that is described in Section 3.
In this section our aim is twofold: first, we characterize the set of possible payoffs of weak SPEs and second, we show how it is possible to build a weak SPE given a set of lassoes with some “good properties”. Those characterizations work for Boolean games with prefix-independent gain functions. We make this hypothesis all along Section 3.
Let be an initialized Boolean game with prefix-independent gain functions. The computation of the set of all the payoffs of weak SPEs in is inspired by a fixpoint procedure explained in [Bruyere0PR17]. Each vertex is labeled by a set of payoffs . Initially, these payoffs are those for which there exists a play in with payoff . Then step by step, some payoffs are removed for the labeling of as soon as we are sure they cannot be the payoff of in a subgame for some weak SPE . (Notice that the value of is not important since the gain functions are prefix independent. This is why we only focus on and not on .) When a fixpoint is reached, the labeling of the initial vertex exactly contains all the payoffs of weak SPEs in .
We formally proceed as follows. For all , we define the initial labeling of as:
Then for each step , we compute the set by alternating between two operations: Remove and Adjust. To this end, we need to introduce the notion of -labeled play. Let be a payoff and be a step, a play is -labeled if for all we have , that is, visits only vertices that are labeled by at step . We first give the definition of the Remove-Adjust procedure and then give some intuition about it.
Definition 6 (Remove-Adjust procedure).
is odd, process theRemove operation:
If for some there exists and such that for all , then and for all , .
If such a vertex does not exist, then for all .
If is even, process the Adjust operation:
If some payoff was removed from (that is, ), then
For all such that , check whether there still exists a -labeled play with payoff from . If it is the case, then , otherwise .
For all such that :
Otherwise for all .
Let us explain the Remove operation. Let that labels vertex . This means that it is the payoff of a potential subgame outcome of a weak SPE that starts in . Suppose that is a vertex of player and has a successor such that for all labeling . Then cannot be the payoff of in the subgame for some weak SPE and some history , otherwise player would have a profitable (one-shot) deviation by moving from to in this subgame.
Now it may happen that for another vertex having in its labeling, all potential subgame outcomes of a weak SPE from with payoff necessarily visit vertex . As has been removed from the labeling of , these potential plays do no longer survive and is also removed from the labeling of by the Adjust operation.
We can state the existence of a fixpoint of the sequences , , in the following meaning:
Proposition 2 (Existence of a fixpoint).
There exists an even natural number such that for all , .
We illustrate the different steps of the Remove-Adjust procedure on the example depicted in Figure 1, and we display the result of this computation in Table 2. Initially, the sets , , contains all payoffs such that there exists a play with . At step , we apply a Remove operation to (this is the only possible ): is a vertex of player and has a successor such that . Therefore is removed from to get . By definition of the Remove operation, the other sets are not modified and are thus equal to . At step , we apply an Adjust operation. The only way to obtain the payoff from is by visiting with the play . As there does not exist a -labeled play with payoff anymore, we have to remove from . The other sets remain unchanged. At step , the Remove operation removes payoff from due to the unique payoff in . At step , the Adjust operation leaves all sets unchanged. Finally at step , the Remove operation also leaves all sets unchanged, and the fixpoint is reached.
Characterization and good symbolic witness
The fixpoint , , provides a characterization of the payoffs of all weak SPEs as described in the following theorem. This result is in the spirit of the classical Folk Theorem which characterizes the payoffs of all NEs in infinitely repeated games (see for instance [fudenberg1991game, Chapter 8]).
Theorem 1 (Characterization).
Let be an initialized Boolean game with prefix-independent gain functions. Then there exists a weak SPE with payoff in if and only if for all and .
In this theorem, only sets with are considered. Indeed subgames of deals with histories , that is, with vertices reachable from . The proof of Theorem 1 uses the concept of (good) symbolic witness defined hereafter. Some intuition about it is given after the definitions.
Definition 7 (Symbolic witness).
Let be an initialized Boolean game with prefix-independent gain functions. Let be the set
A symbolic witness is a set such that each is a lasso of with and with length bounded by .
A symbolic witness has thus at most lassoes (by definition of ) with polynomial length.
Definition 8 (Good symbolic witness).
A symbolic witness is good if for all , for all vertices such that and , we have .
Let us now give some intuition. A strategy profile in induces an infinite number of subgame outcomes , . A symbolic witness is a compact representation of . It is a finite set of lassoes that represent some subgame outcomes of : the lasso of represents the outcome , and its other lassoes represents the subgame outcome for some particular histories . The index records that player can move from (the last vertex of ) to (with the convention that for the outcome ). When is a weak SPE, the related symbolic witness is good, that is, its lassoes avoid profitable one-shot deviations between them.
We come back to our running example. The weak SPE of Example 2 depicted in Figure 1 has payoff . A symbolic witness of is given in Table 3 which is here composed of all the subgame outcomes of . One can check that is a good symbolic witness. For instance, consider its lassoes and , the vertex of and the edge . We have . Indeed in the subgame , player has no profitable one-shot deviation by using the edge .
Theorem 1 is a direct consequence of the next proposition.
Let be an initialized Boolean game with prefix-independent gain functions. The following assertions are equivalent:
There exists a weak SPE with payoff in ;
for all and
There exists a good symbolic witness that contains a lasso with payoff ;
There exists a finite-memory weak SPE with payoff in such that the size of each strategy is in .
Let us give a sketch of proof. For , if there exists a weak SPE , then for all and all , the payoff survives in after both a Remove and an Adjust operation. This shows that the fixpoint is non empty. For , if each contains some payoff , then there exists a -labeled play such that , and this play can be supposed to be a lasso of size at most by [BouyerBMU15, Proposition 3.1]. We then construct a symbolic witness composed of some of those lassoes: is a lasso extracted from , and each other lasso is extracted from such that its payoff satisfies . The way the fixpoint is constructed guarantees that if with , we have (by the Remove operation). The latter property implies that is good. For , from the existence of a good symbolic witness , it is possible to construct a strategy profile such that its subgame outcomes are the lassoes of , and that is a weak SPE because is good. Moreover the strategies of are finite-memory with size in because is constructed from at most lassoes with length bounded by . Finally is immediate.
Recall that Proposition 3 only works for prefix-independent gain functions. Nevertheless a Boolean game with Reachability (resp. Safety) objectives can be transformed into a Boolean game with Büchi (resp. Co-Büchi) objectives. The construction is classical: it stores inside the vertices the set of players who have already visited , where is the set they want to visit (resp. to avoid). Consequently we have the next result from equivalence of Proposition 3.
Let be an initialized Boolean game. There exists a weak SPE in if and only if there exists a finite-memory weak SPE with the same payoff. Moreover, the size of each strategy is
in for Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, and Streett objectives,
in for Reachability and Safety objectives.
4 Complexity of the constraint problem
In this section, we first study the complexity classes of the constraint problem for Boolean games with all classical -regular objectives (except Explicit Muller objectives). The concept of good symbolic witness is essential in this study. Second, we show that the constraint problem is -complete for Explicit Muller objectives, that it is fixed parameter tractable for the other classical -regular objectives, and that it becomes polynomial when the number of players is fixed. Those results rely on the fixpoint algorithm given in the previous section.
Complexity classes of the constraint problem
The exact complexities of the constraint problem are stated in the following theorem.
The constraint problem for Boolean games
is PSPACE-complete for Reachability and Safety objectives;
is NP-complete for co-Büchi, Parity, Muller, Rabin, and Streett objectives; it is in NP for Büchi objectives.
Let us give a sketch of proof. For Part 2 of Theorem 2, as the considered objectives are prefix-independent, we can apply Proposition 3, in particular equivalence . To prove NP-membership, given thresholds , a nondeterministic polynomial algorithm works as follows: guess a set composed of at most lassoes of length bounded by and check that is a good symbolic witness that contains a lasso with payoff such that . The NP-hardness is obtained thanks to a polynomial reduction from SAT, using exactly the same reduction as proposed in [Ummels08] for the constraint problem for NEs in Boolean games with co-Büchi objectives. Notice that this reduction does not work for Büchi objectives. For Part 1 of Theorem 2, we transform a given Boolean game with Reachability (resp. Safety) objectives into a Boolean game with Büchi (resp. Co-Büchi) objectives (as explained at the end of Section 3
). With this transformation, a good symbolic witness has an exponential number of lassoes however still with polynomial size. To prove PSPACE-membership, we here consider an alternating Turing machine such that along each execution, the playersand play in a turned based way during a polynomial number of turns: player proposes lassoes (taken in a good symbolic witness if it exists) and player tries to show that is not good (by proposing a deviation between two lassoes that do not respect Definition 8). This Turing machine works in polynomial time, and as PSPACE APTIME, we get PSPACE-membership. The PSPACE-hardness is obtained with a polynomial reduction from QBF. This reduction is more involved than the one used to prove NP-hardness. Indeed the reduction for NP-hardness already works for NEs whereas the reduction for PSPACE-hardness really exploits the subgame perfect aspects.
Notice that the constraint problem for SPEs (instead of weak SPEs) for Boolean games with Reachability and Safety objectives is also PSPACE-complete. Indeed one can prove that weak SPEs and SPEs are equivalent notions for Reachability objectives. The case of Safety objectives needs another approach, it results from the proof of Theorem 2.
The constraint problem for SPEs in Boolean games with Reachability and Safety objectives is PSPACE-complete.
Fixed parameter tractability and Explicit Muller objectives
We now study the complexity of the constraint problem for Explicit Muller objectives, and its fixed parameter tractability for the other objectives. We recall that a parameterized language is a subset of , where is a finite alphabet, the second component being the parameter of the language. It is called fixed parameter tractable if there is an algorithm that determines whether in time, where is a constant independent of the parameter and is a computable function depending on only. We also say that belongs to the class FPT. Intuitively, a language is in FPT if there is an algorithm running in polynomial time with respect to the input size times some computable function on the parameter. We refer the interested reader to [DowneyF99] for more details on parameterized complexity.
The constraint problem for multiplayer Boolean games with Explicit Muller objectives is P-complete.
Let be a Boolean game.
The constraint problem is in FPT for Reachability, Safety, Büchi, co-Büchi, Parity, Muller, Rabin, and Streett objectives. The parameters are
the number of players for Reachability, Safety, Büchi, co-Büchi, and Parity objectives,
the number of players and the numbers , , of pairs , for Rabin and Streett objectives, and
the number of players, the numbers , , of colors and the sizes , , of the families of subsets of colors for Muller objectives.
When the number of players is fixed, for all these kinds of objectives, the constraint problem can be solved in polynomial time.
Notice that in Theorem 4, to obtain fixed parameter tractability for Rabin, Streett, and Muller objectives, in addition to the number of players, we also have to consider the parameter equal to the size of the objective description. Nevertheless, when the number of players is fixed, we get polynomial time complexity for all types of objectives.
The results of Theorems 3 and 4 do not rely on the concept of good symbolic witness (as for Theorem 2) but rather on the following algorithm based on Theorem 1 and called the decision algorithm. Given a Boolean game and thresholds ,
Compute the initial sets , , and repeat the Remove-Adjust procedure (see Definition 6) until reaching the fixpoint , ,
Then check whether for all