1 Introduction
Over the last years, Stackelberg games are receiving an increasing attention from the algorithmic game theory community, thanks to their many applications in realworld scenarios, such as in security
[Tambe2011]. In the classical Stackelberg setting [Von Stackelberg1934], there is a leader with the ability to play before the other player, who acts as follower by observing the realization of the leader’s strategy. In this work, we follow a different approach, where the leader looks for a strategy to commit to [Conitzer and Sandholm2006], and the follower observes the leader’s mixed strategy, without knowing its actual realization. An interpretation of this setting is provided by von2010leadership [von2010leadership]: any (underlying) game is extended as a sequential game in which the leader plays first, having a continuum of choices corresponding to mixedstrategy commitments.The majority of the works in the literature focus on games with a single leader and a single follower [Conitzer and Sandholm2006, Von Stengel and Zamir2010]. In this setting, the leader seeks for a utilitymaximizing mixed strategy to commit to, while the follower plays a best response to the commitment. This model has been largely studied, especially for security applications [Paruchuri et al.2008, Kiekintveld et al.2009, An et al.2011].
Some works also study games with a single leader and multiple followers. conitzer2011commitment [conitzer2011commitment] introduce a model where the leader commits to a correlated strategy and, accordingly, she draws recommendations for the followers, who must obey the incentive constraints of correlated equilibria [Aumann1974]. The authors show that, in normalform games, an optimal correlated strategy to commit to can be computed in polynomial time. Other works study situations where the followers play a Nash equilibrium [Nash1951] in the game resulting from the leader’s mixedstrategy commitment [Von Stengel and Zamir2010, Coniglio et al.2017, Coniglio et al.2018, De Nittis et al.2018, Marchesi et al.2018, Marchesi et al.2019]. However, these models usually result in intractable computational problems even with a fixed number of followers.
Settings including multiple leaders are widely unexplored in the literature. In spite of this, many realworld applications naturally involve more than one player with competitive advantages, playing the role of leader. Some scenarios are, e.g., network platforms with premium (prioritized) users, markets where a group of firms forms a pricedetermining dominant cartel, and political elections in which some candidates choose policy positions in advance of challengers.
Restricted to the security context, there are some works addressing games with multiple uncoordinated defenders (leaders) [Smith et al.2014, Lou and Vorobeychik2015, Laszka et al.2016, Lou et al.2017, Gan et al.2018]. However, differently from our work, they all enforce Nashlike constraints on the leaders’ strategies. Moreover, their models suffer from two major drawbacks: (i) an exact equilibrium may not exist, and (ii) they strongly rely on problemspecific structures arising in security problems.
The operations research literature provides further works on multileaderfollower settings, under the name of mathematical programs with equilibrium constraints [Luo et al.1996]. They assume that both leaders and followers are subject to Nash constraints, with the latter playing in the game resulting from the leaders’ strategies [Leyffer and Munson2010, Kulkarni and Shanbhag2014]. Furthermore, other works from the same field focus on oligopoly models where the leaders select the level of investment to maximize profits [DeMiguel and Xu2009]. All these works considerably depart from ours, as they use fundamentally different models and lack thorough game theoretic and computational studies.
1.0.1 Original Contributions
We introduce a new way to apply the Stackelberg paradigm to any finite (underlying) game. Our approach extends the idea of commitment to correlated strategies in settings involving multiple leaders and followers, generalizing the work of conitzer2011commitment [conitzer2011commitment]. The crucial component of our framework is that a leader can decide whether to participate in the commitment or to defect from it by becoming a follower. This induces a preliminary agreement stage that takes place before the underlying game is played, where the leaders decide, in turn, whether to opt out from the commitment or not. We model this stage as a sequential game, whose size is factorial in the number of players. Our goal is to identify commitments guaranteeing some desirable properties that we define on the agreement stage. The first one requires that the leaders do not have any incentive to become followers. It comes in two flavors, called stability and perfect stability, which are related to, respectively, Nash and subgame perfect equilibria of the sequential game representing the agreement stage. The second property is also defined in two flavors, namely efficiency and perfect efficiency, both enforcing Pareto optimality with respect to the leaders’ utility functions, though at different levels of the agreement stage.
We introduce three solution concepts, which we generally call Stackelberg correlated equilibria (SCEs). They differ depending on the properties they call for. Specifically, SCEs, SCEs with perfect agreement (SCEPAs), and SCEPAs with perfect efficiency (SCEPAPEs) require, respectively, stability and efficiency, perfect stability and efficiency, and both perfect stability and perfect efficiency.
First, we investigate the game theoretic properties of our solution concepts. We show that SCEs and SCEPAs are guaranteed to exist in any game, while SCEPAPEs may not. Moreover, we compare them with other solution concepts.
Then, we switch the attention to the computational complexity perspective. We show that, provided a suitably defined stability oracle is solvable in polynomial time, an SCE optimizing some linear function of leaders’ utilities (such as the leaders’ social welfare) can be computed in polynomial time, even in the number of players. The same holds for finding an SCEPA, while we prove that computing an optimal SCEPA is an intractable problem. Nevertheless, in the latter case, we provide an (exponential in the game size) upper bound on the necessary number of queries to the oracle.
In conclusion, we study which classes of games admit a polynomialtime stability oracle, focusing on those with polynomial type [Papadimitriou and Roughgarden2008]. We show that the problem solved by our oracle is strictly connected with the weighted deviationadjusted social welfare problem introduced by leyton2011ellipsoid [leyton2011ellipsoid]. As a result, we get that our oracle is solvable in polynomial time in all the game classes where the same holds for the problem of finding an optimal correlated equilibrium. ^{2}^{2}2Full proofs of all the results are in Appendices B, C, D, and E.
2 Preliminaries
In this section, we introduce some basic concepts about games and their equilibria used in the rest of the paper.
2.1 Finite Games and Their Equilibria
A (finite) game is a tuple , where is a finite set of players, is a finite set of player ’s strategies or actions, and is player ’s utility, defined over the set of strategy profiles . Given , let be the partial profile obtained by dropping player ’s strategy from , so that . We call the set of correlated distributions defined over strategy profiles, i.e., each satisfies and for all . Moreover, overloading notation, is player ’s expected utility in .
A correlated distribution is a correlated equilibrium (CE) [Aumann1974] if, for every player and strategies , the following constraint holds:
(1) 
We can interpret a CE in terms of a mediator who draws some strategy profile from a publicly known distribution , and, then, it privately communicates each recommendation to player . The distribution is an equilibrium if no player has an incentive to deviate from the recommendation, as made formal by the incentive constraints of Eq. (1). Moreover, a Nash equilibrium (NE) [Nash1951] is a CE that can be written as the product distribution of players’ mixed strategies, i.e., for all , where each
is a probability distribution over strategies
denoting a player ’s mixed strategy.In the following, we denote with the set of correlated distributions that satisfy the incentive constraints of Eq. (1) only for a subset of players . Clearly, is the set of CEs of the game.
Different classes of games are employed depending on how strategies and utilities are represented. The most common representation is the normal form, which encodes each utility function as an dimensional matrix indexed by . Thus, the size of a normalform game is exponential in the number of players. Many other representations have been introduced in the literature. In Section 7, we are interested in those with polynomial type [Papadimitriou and Roughgarden2008], where the number of players and the number of strategies are bounded by polynomials in the size of the game. Many important classes of games admit a polynomialtype representation, such as graphical games [Kearns et al.2013], polymatrix games [Eaves1973], anonymous games [Blonski2000], and congestion games [Rosenthal1973].
2.2 Stackelberg Games and Equilibria
Any finite game has a Stackelberg counterpart where some of the players are leaders and the others are followers. The former have the ability to commit to a course of play beforehand, while the latter decide how to play after observing the commitment [Von Stengel and Zamir2010].
Definition 1.
Given a finite game , a Stackelberg game (SG) is a tuple where and are the sets of leaders and followers, respectively, with .
In singleleader singlefollower SGs, the follower best responds to the leader’s mixedstrategy commitment [Conitzer and Sandholm2006, Von Stengel and Zamir2010].
Definition 2.
Given an SG , a leader’s mixed strategy defines a Stackelberg equilibrium (SE) if it maximizes given that, for each , the follower plays an maximizing . ^{3}^{3}3In the literature, different SE concepts are defined depending on how the follower is assumed to break ties. The strong and weak SEs are two notable cases [Breton et al.1988], where the follower is assumed to break ties in favor and against the leader, respectively.
The multifollower case unfolds in different scenarios depending on how the followers are assumed to play. conitzer2011commitment [conitzer2011commitment] study what they call optimal correlated strategies to commit to, where the leader commits to a utilitymaximizing correlated distribution satisfying the incentive constraints (Eq. (1)) for the followers only. Formally:
Definition 3.
Given an SG , is an optimal correlated strategy to commit to if it maximizes the leader’s utility over the set .
In our work, we pursue the approach of conitzer2011commitment [conitzer2011commitment], rather than letting the followers play an NE, as done, e.g., by von2010leadership [von2010leadership]. Indeed, while the two models provide the same leader’s utility in singlefollower SGs (corresponding to that in an SE), the latter may be strictly better in SGs with two or more followers (see [Conitzer and Korzhyk2011] for an example).
3 MultiLeaderFollower Stackelberg Games
We address SGs with multiple leaders and followers. The key components of our approach are the following. First, we allow the leaders to decide whether to participate in the commitment or to defect from it by taking on the role of followers. This is modeled by the agreement stage of the SG, whose result is the formation of an agreement involving a subset of the leaders. Second, in the spirit of CEs, we introduce a correlation device that, after the agreement, draws recommendations and privately communicates them to the players. Following conitzer2011commitment [conitzer2011commitment], we assume that the leaders involved in the agreement commit to play their recommendations, while the followers obey to the usual incentive constraints of CEs (see Eq. (1)). The correlation device may adopt different distributions depending on the sequence of defections that determined the agreement, and these distributions are publicly known. Our goal is to design the device, so as to achieve some desirable properties of the commitment, which we formally describe in the rest of the section.
Before going into our main definitions, we introduce some useful notation. Given a subset of players , we denote with the collection of ordered subsets of , including the empty set . Given and , we let be the ordered set obtained by appending at the end of . We use
to denote a vector of correlated distributions
, one per ordered subset of leaders , while is the set of all such vectors. In words, represents a sequence of leaders’ defections in the agreement stage, while defines the publicly known correlated distributions adopted by the correlation device, with being the one used when the sequence of defections is .Definition 4.
Given a vector of distributions , an SG is structured in the following two stages:

[nolistsep,itemsep=0mm]

Agreement. It goes on in rounds. In a given round, each leader, in turn, decides between OptIn and OptOut.^{4}^{4}4We assume that the leaders are asked to take a decision according to some ordering, e.g., decides before if . All the decisions are perfectly observable. If a player chooses OptOut, then she leaves the set of leaders becoming a follower, and a new round starts. The stage ends when, during a round, all remaining leaders decide to OptIn. The result is the ordered subset of leaders who decided to OptOut. ^{5}^{5}5The agreement stage is finite as there are at most rounds and each round involves at most decisions. ^{6}^{6}6Our results do not rely on the protocol implemented in the agreement stage. Others could be adopted, with the only requirement that they must record in which order the leaders do OptOut.

Play. The correlation device draws some according to the publicly known correlated distribution . Then, each player is privately told her recommendation and the underlying game is played, with the leaders in sticking to their recommendations.
Remark 1.
The agreement stage of an SG can be represented as a sequential (i.e., treeform) game involving the leaders. In such game, the players play in turn, according to some fixed order, with only two actions available at each decision point: OptIn and OptOut. When a player chooses OptOut, then she never plays anymore. The game ends after a sequence of OptIn actions performed by all leaders who have not selected OptOut yet. Thus, each leaf of the game corresponds to the ordered subset representing the sequence of leaders who performed OptOut on the path to the leaf. Players’ payoffs are defined by for . See Figure 1 (Right) for an example of sequentialgamerepresentation of the agreement stage.
Next, we introduce some desirable properties that the distributions of the correlation device should satisfy. In the following definitions, we assume that an SG is given.
First, we introduce stability. In words, we require that the leaders in do not have any incentive to become followers. We introduce two different notions of stability, as follows.
Definition 5.
Given , for any , is stable if, for every , . Moreover:

[nolistsep,itemsep=0mm]

is stable if is stable;

is perfectly stable if is stable for every .
We denote with and the sets of stable and perfectly stable distributions, respectively.
Remark 2.
The rationale behind stability is that of NE. Indeed, is stable if and only if each leader playing OptIn is an NE of the sequential game representing the agreement stage. Intuitively, this is because, if is stable, each leader must not have any incentive to play OptOut given that the other leaders always play OptIn.
Remark 3.
The rationale behind perfect stability is that of subgame perfection. Indeed, is perfectly stable if and only if each leader playing OptIn is a subgame perfect equilibrium of the agreement stage. The reason is that perfect stability requires that playing OptIn is optimal at any decision point of the sequential game.
The second property that we look for is efficiency. We require that the correlated distributions of the correlation device are Pareto optimal with respect to the utility functions of the leaders who decided to OptIn. Given , for , we use to denote the set of Pareto optimal correlated distributions in the set , where the objectives are the functions , for . Formally:
Definition 6.
Given , for any , is efficient on the set if . Moreover:

[nolistsep,itemsep=0mm]

is efficient on if is efficient on ;

is perfectly efficient on if is efficient on for every .
We introduce three different solution concepts for our SGs, which we refer to as Stackelberg correlated equilibria (SCEs). They differ on the types of stability and efficiency that they prescribe. Formally:
Definition 7.
Given an SG , is an:

[nolistsep,itemsep=0mm]

SCE if it is efficient on the set ;

SCE with perfect agreement (SCEPA) if it is efficient on the set ;

SCE with perfect agreement and perfect efficiency (SCEPAPE) if it is perfectly efficient on the set .
We denote with , , and the sets of SCEs, SCEPAs, and SCEPAPEs, respectively.
Example 1.
Consider the SG in Figure 1, where and . Let be such that , , and for all the other . Clearly, for all . Moreover, being stable and Pareto optimal, is an SCE. Observe that, if player 2 performs OptOut, prescribes an irrational behavior to player 1, as , while she gets by doing OptOut. Thus, is not perfectly stable, as playing OptIn must be optimal at any decision point of the agreement stage. For instance, with and for every other is an SCEPA. However, notice that is not an SCEPAPE since does not maximize player 1’s utility. Instead, with , , and for all the other is an SCEPAPE.
4 On the Existence of SCEs
We investigate the existence of our solution concepts in general SGs. We show that SCEs and SCEPAs always exist, while we provide an SG where there is no SCEPAPE.
The fundamental step for proving our existence results (Theorem 1) is to show that (i) and are polytopes, and (ii) they are nonempty. The latter point is a direct consequence of the fact that all vectors with for some CE are perfectly stable.
Theorem 1.
Every SG admits an SCE and an SCEPA.
Proposition 1.
There are SGs with no SCEPAPE.
Proof sketch.
Consider the SG in Table 1, where and . Any must be such that, for every with player 3 in , (as player 3 always gets by deviating to ). Given the definition of stability and player 3’s incentive constraints, and must always recommend to player 3. Moreover, by stability and efficiency, must always recommend , where player 1 gets a utility of . Similarly, must always recommend and, thus, player 2 receives a utility of . Thus, for stability, must satisfy , which is impossible. ∎
As a result, in the rest of this work we focus on SCEs and SCEPAs. We remark that the nonexistence of SCEPAPEs implies that, under the requirements of perfect stability and perfect efficiency, there cannot be an agreement involving all the leaders. This does not rule out the possibility that some subsets of leaders can still reach an agreement. However, these cases are much more involved, as the actual group of leaders reaching an agreement inevitably depends on the rules of the protocol implemented in the agreement stage.
5 SCEs and Other Solution Concepts
We show that the optimal correlated strategies to commit to introduced by conitzer2011commitment [conitzer2011commitment] are a special case of SCEs. Intuitively, in singleleader SGs, efficiency is equivalent to the maximization of leader’s utility, while stability does not enforce additional constraints on the commitment.
Theorem 2.
Given an SG , it holds and, given some , is an optimal correlated strategy to commit to.
Proof sketch.
Since the SG has only one leader (player 1), , and, thus, . For the same reasons, . Moreover, Pareto optimality is the same as maximizing the leader’s utility function . Let and assume, by contradiction, that is not an optimal correlated strategy to commit to. Then, there would be another such that . However, replacing with in would give us another , contradicting the efficiency of . ∎
Given the relation between optimal correlated strategies to commit to and SEs in singleleader singlefollower SGs:
Corollary 2.1.
Given an SG , any is such that is the leader’s utility in an SE.
For the relationships of SCEs with other nonStackelberg solution concepts, we refer the reader to Appendix A.
6 Computational Complexity of SCEs
We study the computational complexity of SCEs and SCEPAs in general SGs. We distinguish between the problem of finding an equilibrium and that of computing an optimal equilibrium, i.e., one maximizing a specific given linear function of leaders’ utilities, such as the leader’s social welfare. We introduce the following formal definitions (problems fSCEPA and oSCEPA() are defined analogously for SCEPAs).
Definition 8 (fSCE).
Given an SG , find an SCE.
Definition 9 (oSCE()).
Given an SG and , find an SCE maximizing the objective function .
Let us remark that, in general, the size of a vector is factorial in the number of players. Thus, in the following, we assume that there is some compact representation for . ^{7}^{7}7As we see next, for all our positive results we can safely assume that there is a compact representation for (e.g., only requires a polynomial number of polynomiallysized distributions).
We establish a tight connection between our problems and an auxiliary one, which is a generalization of the problem of finding an optimal CE. In the rest of the section, we assume to have access to an oracle solving this auxiliary problem, which we call stability oracle. In Section 7, we then investigate for which games the oracle can be efficiently implemented.
Definition 10.
A stability oracle is an algorithm that, given a finite game , a coefficients vector , a set of leaders , and a collection of correlated distributions for , returns an maximizing subject to the stability constraints, i.e., for all . ^{8}^{8}8Note that, given a finite game , returns an optimal CE for the objective function defined by .
In the following, we are interested in games where the stability oracle runs in polynomial time. Thus, we assume that always returns a correlated distribution with size polynomial in the size of the game. ^{9}^{9}9Indeed, this assumption is not restrictive, as all the games we study in Section 7 admit a polytime oracle with this property. We also consider the decision form of the stability oracle, which reads as follows:
Definition 11.
The decision form of a stability oracle is an algorithm that, given , , and for , answers Yes if and satisfies the stability constraints, and No otherwise.
In the following, given and , we let be such that if , while if not. Moreover, given , we let be such that and for all . Note that is the coefficients vector of the objective , while corresponds to minimizing .
6.1 Computing SCEs
We show that, in games admitting a polynomialtime stability oracle, an optimal SCE can be computed in polynomial time. Intuitively, oSCE() is solved by computed as: for , , and for every other ordered subset . Formally:
Theorem 3.
Given an SG and , oSCE() can be solved with queries to an oracle .
Corollary 3.1.
Given an SG , if there is a polytime oracle , then oSCE() can be solved in polynomial time.
6.2 Computing SCEPAs
First, we provide a positive result: one can find an SCEPA with polynomially many invocations to a stability oracle. It is sufficient to compute where for and . Thus:
Theorem 4.
Given an SG , fSCEPA can be solved with queries to an oracle .
Corollary 4.1.
Given an SG , if there is a polytime oracle , then fSCEPA can be solved in polynomial time.
Now, we switch to the problem of computing an optimal SCEPA, showing that it cannot be solved efficiently, even with access to a polynomialtime stability oracle. Specifically, we prove a stronger negative result: even the easier problem of verifying the perfect stability of a given is computationally intractable. Our statement is based on a reduction from the coNPcomplete problem of deciding whether a given formula in disjunctive normal form (DNF) is a tautology or not [Arora and Barak2009].
Theorem 5.
Given an SG and , verifying whether or is not in P unless NP = coNP, even with access to a polynomialtime decisionform oracle .
Proof sketch..
Given a formula in DNF, we construct an SG involving a leader for each variable and a single follower . Each has two actions, and , which define the truth value of . As a result, any corresponds to a truth assignment defined by leaders’ strategies. The follower has a strategy for each variable . Table 2 reports the leaders’ utilities (the follower always gets ). We build with for some such that for all . Furthermore, for every and , we let for with for all , for all , and .
If. Suppose is a tautology. For every , recommends all the leaders in to play . Note that, for every and , , while, if decides to OptOut, she is recommended to play (one more variable is set to false) and, being , she gets the same utility. As a result, all distributions are stable.
Only if. Suppose is not a tautology. Let be such that . If for every , then is not stable as the leaders would OptOut (getting at least ). Otherwise, there exists such that and , for some , . Then, and . Thus, is not stable, as leader would OptOut. ∎
As a byproduct of Theorem 3 we have that, when looking for optimal SCEs, one can restrict the attention to those admitting a representation whose size is polynomial in the size of the game. For Theorem 4, the same holds when searching for an SCEPA. However, Theorem 5 implies that optimal SCEPAs require an exponential number of different distributions. Moreover, even when can be easily represented in a compact form (as in the proof of Theorem 5), we cannot check in polynomial time whether or not.
This poses a new intriguing question: can we restrict the attention to whose size is less than factorial in the number of players? We show that the answer is positive. It is sufficient to consider whose size is exponential in the number of players, as only the unordered set of defecting leaders and the last of them who decided to OptOut matter.
Theorem 6.
Given an SG and , there is an s.t. and for every and defining the same set.
Theorem 6 allows us to reduce the number of queries to a stability oracle that are necessary to find an optimal SCEPA.
Theorem 7.
Given an SG and , oSCEPA() can be solved with queries to .
Finally, we can provide an example showing that Theorem 7 is tight, which leads to the following proposition.
Proposition 2.
Solving oSCEPA() requires to take into account the last player who performed OptOut, while focusing only on the set of defecting leaders is not sufficient.
7 Stability Oracle for Compact Games
We study which classes of games admit a polynomialtime stability oracle , focusing on those with polynomial type. ^{10}^{10}footnotemark: 10
In this section, we only provide our main final result; a detailed description of all the ancillary results is in Appendix E.
Inspired by the classical approaches for finding CEs in games with polynomial type [Papadimitriou and Roughgarden2008, Jiang and LeytonBrown2011, Jiang and LeytonBrown2015], we solve in polynomial time using the ellipsoid method. This requires that a suitably defined separation problem (Sep) can be computed in polynomial time. Our main result is that Sep can be reduced to the weighted deviationadjusted social welfare problem (wDaSW) introduced by leyton2011ellipsoid [leyton2011ellipsoid] for finding an optimal (according to some linear function of players’ utilities) CE. This establishes a strict connection between the problem solved by our stability oracle and that of computing optimal CEs. As a consequence, given the results of leyton2011ellipsoid [leyton2011ellipsoid], can be computed in polynomial time for all the compact games where finding an optimal CE is computationally tractable. Thus:
Theorem 8.
The following games admit a polynomialtime stability oracle : anonymous games, symmetric games, and boundedtreewidth graphical and polymatrix games.
8 Discussion
This paper introduces a new way to apply the Stackelberg paradigm to any (underlying) finite game. Differently from previous works, our approach deals with scenarios involving multiple leaders by introducing a preliminary agreement stage in which each leader can decide whether to be a leader or become a follower. We introduce and study three natural solution concepts that differ depending on the properties that they require on the agreement stage (others, e.g., requiring stability and perfect efficiency, will be explored in future).
Our equilibria generalize the optimal correlated strategies to commit to introduced by conitzer2011commitment [conitzer2011commitment] for singleleader multifollower Stackelberg games. At the same time, they also provide a significant advancement over the multileader solution concepts introduced in the security context (see, e.g., [Gan et al.2018]). First, correlatedstrategy commitments are more natural than leaders’ strategies satisfying some Nashlike constraints. Second, our equilibria are funded on strong gametheoretic groundings, as they are guaranteed to exist independently of the game structure. Last but not least our solutions apply to general games.
Finally, our computational findings exploit a general framework relying on a gameindependent stability oracle. Thus, our positive results can be extended to other game classes by simply designing polynomialtime oracles.
Acknowledgments
This work has been partially supported by the Italian MIUR PRIN 2017 Project ALGADIMAR “Algorithms, Games, and Digital Market”.
References
 [An et al.2011] B. An, J. Pita, E. Shieh, M. Tambe, C. Kiekintveld, and J. Marecki. Guards and Protect: Next generation applications of security games. ACM SIGEC EX, 10(1):31–34, 2011.
 [Arora and Barak2009] S. Arora and B. Barak. Computational complexity: a modern approach. 2009.
 [Aumann1974] R.J. Aumann. Subjectivity and correlation in randomized strategies. J MATH ECON, 1(1):67–96, 1974.
 [Blonski2000] M. Blonski. Characterization of pure strategy equilibria in finite anonymous games. J MATH ECON, 34(2):225–233, 2000.
 [Breton et al.1988] M. Breton, A. Alj, and A. Haurie. Sequential stackelberg equilibria in twoperson games. J OPTIMIZ THEORY APP, 59(1):71–97, 1988.
 [Coniglio et al.2017] S. Coniglio, N. Gatti, and A. Marchesi. Pessimistic leaderfollower equilibria with multiple followers. In IJCAI, 2017.
 [Coniglio et al.2018] Stefano Coniglio, Nicola Gatti, and Alberto Marchesi. Computing a pessimistic leaderfollower equilibrium with multiple followers: the mixedpure case. CoRR, abs/1808.01438, 2018.
 [Conitzer and Korzhyk2011] V. Conitzer and D. Korzhyk. Commitment to correlated strategies. In AAAI, 2011.
 [Conitzer and Sandholm2006] V. Conitzer and T. Sandholm. Computing the optimal strategy to commit to. In ACM EC, 2006.
 [De Nittis et al.2018] G. De Nittis, A. Marchesi, and N. Gatti. Computing the strategy to commit to in polymatrix games. In AAAI, 2018.
 [DeMiguel and Xu2009] V. DeMiguel and H. Xu. A stochastic multipleleader stackelberg model: analysis, computation, and application. OPER RES, 57(5):1220–1235, 2009.
 [Eaves1973] B.C. Eaves. Polymatrix games with joint constraints. SIAM J APPL MATH, 24(3):418–423, 1973.
 [Gan et al.2018] J. Gan, E. Elkind, and M. Wooldridge. Stackelberg security games with multiple uncoordinated defenders. In AAMAS, 2018.
 [Jiang and LeytonBrown2011] A.X. Jiang and K. LeytonBrown. A general framework for computing optimal correlated equilibria in compact games. In WINE, 2011.
 [Jiang and LeytonBrown2015] A.X. Jiang and K. LeytonBrown. Polynomialtime computation of exact correlated equilibrium in compact games. GAME ECON BEHAV, 91:347–359, 2015.
 [Kearns et al.2013] M. Kearns, M. Littman, and S. Singh. Graphical models for game theory. arXiv:1301.2281, 2013.
 [Kiekintveld et al.2009] C. Kiekintveld, M. Jain, J. Tsai, J. Pita, F. Ordóñez, and M. Tambe. Computing optimal randomized resource allocations for massive security games. In AAMAS, 2009.
 [Kulkarni and Shanbhag2014] A.A. Kulkarni and U.V. Shanbhag. A sharedconstraint approach to multileader multifollower games. SETVALUED VAR ANAL, 22(4):691–720, 2014.
 [Laszka et al.2016] A. Laszka, J. Lou, and Y. Vorobeychik. Multidefender strategic filtering against spearphishing attacks. In AAAI, 2016.
 [Leyffer and Munson2010] S. Leyffer and T. Munson. Solving multileadercommonfollower games. OPT MET SO, 25(4):601–623, 2010.
 [Lou and Vorobeychik2015] J. Lou and Y. Vorobeychik. Equilibrium analysis of multidefender security games. In IJCAI, 2015.
 [Lou et al.2017] J. Lou, A.M. Smith, and Y. Vorobeychik. Multidefender security games. IEEE INTELL SYST, 32(1):50–60, 2017.
 [Luo et al.1996] ZQ. Luo, JS. Pang, and D. Ralph. Mathematical programs with equilibrium constraints. 1996.
 [Marchesi et al.2018] A. Marchesi, S. Coniglio, and N. Gatti. Leadership in singleton congestion games. In IJCAI, 2018.
 [Marchesi et al.2019] A. Marchesi, M. Castiglioni, and N. Gatti. Leadership in congestion games: Multiple user classes and nonsingleton actions. In IJCAI, 2019.
 [Moulin and Vial1978] H. Moulin and JP Vial. Strategically zerosum games: the class of games whose completely mixed equilibria cannot be improved upon. INT J GAME THEORY, 7(3):201–221, 1978.
 [Nash1951] J. Nash. Noncooperative games. ANN MATH, pages 286–295, 1951.
 [Papadimitriou and Roughgarden2008] C.H. Papadimitriou and T. Roughgarden. Computing correlated equilibria in multiplayer games. J ACM, 55(3):14, 2008.
 [Paruchuri et al.2008] P. Paruchuri, J. P. Pearce, J. Marecki, M. Tambe, F. Ordonez, and S. Kraus. Playing games for security: an efficient exact algorithm for solving bayesian stackelberg games. In AAMAS, 2008.
 [Rosenthal1973] R.W. Rosenthal. A class of games possessing purestrategy nash equilibria. INT J GAME THEORY, 2(1):65–67, 1973.
 [Shoham and LeytonBrown2008] Y. Shoham and K. LeytonBrown. Multiagent systems: Algorithmic, gametheoretic, and logical foundations. 2008.
 [Smith et al.2014] A. Smith, Y. Vorobeychik, and J. Letchford. Multidefender security games on networks. PERF E R, 41(4):4–7, 2014.
 [Tambe2011] M. Tambe. Security and Game Theory: Algorithms, Deployed Systems, Lessons Learned. 2011.
 [Von Stackelberg1934] H. Von Stackelberg. Marktform und Gleichgewicht. 1934.
 [Von Stengel and Zamir2010] B. Von Stengel and S. Zamir. Leadership games with convex strategy sets. GAME ECON BEHAV, 69(2):446–457, 2010.
Appendix A SCEs and nonStackelberg Correlation
We analyze how our solution concepts relate to other nonStackelberg solutions involving correlation. Specifically, we focus on CEs (see Section 2 for a formal definition) and their coarse variant, which we define in the following.
The coarse CE weakens the CE by only enforcing protection against a priori defections, i.e., before the recommendations are revealed to the players [Moulin and Vial1978]. Formally, is a coarse correlated equilibrium (CCE) if, for every player and strategy , the following constraint holds:
(2) 
We denote with the set of CCEs of the game.
In our analysis, we compare CEs and CCEs with the correlated distributions resulting from our solution concepts in general SGs. Given an SG , we define and as the sets of such that and , respectively. Our goal is to investigate the relationships involving the sets and with the sets of CEs and CCEs of the underlying game , namely and . Figure 2 depicts these relationships.
Let us remark that the relations , , and hold by definition, while it is easy to show that (see the proof of Theorem 1).
First, we look at the connection between (perfectly) stable distributions and CCEs. Given the relation between SEs and SCEs (see Corollary 2.1) in singleleader singlefollower SGs, the following result holds as a direct consequence of [Von Stengel and Zamir2010, Remark 13].
Proposition 3.
There are SGs where .
Moreover, not all perfectly stable distributions are CCEs.
Proposition 4.
There are SGs where .
Proof.
Consider the SG in Table 3 (Left), where and . Since and are strictly dominated, there is a unique CCE with . Let be such that and for all . Notice that each with satisfies the incentive constraints of Eq. (1) for every player, and, thus, . Moreover, for each leader , and for all . Thus, each is stable and . ∎
Next, we analyze the relationships with the sets and , which are defined as and , but for the SG where each player is a leader. Our goal is to study the impact of players’ roles in SGs having the same underlying finite game. The following result shows that enlarging the set of leaders can only introduce new stable distributions.
Theorem 9.
and .
Proof.
We only prove the result for , as similar arguments hold for . Given any SG , for every perfectly stable of , we show that there exists a perfectly stable of , such that . Let us define , for all . Clearly, it holds , as . For every player and such that , we have and . Thus, given that , satisfies the stability constraints for the players in . Now, in order to show that , it is sufficient to prove that players in do not have an incentive to OptOut in . This is the case as, for and with , we have . ∎
Furthermore, we can also provide examples showing that:
Proposition 5.
There are SGs where .
Proof.
Proposition 6.
There are SGs where .
Proof.
Consider the SG in Table 3 (Right), where . There is a CCE with . We show that there is no with . By contradiction, assume there exists such . Given that , it should be the case that , by stability of . Take the incentive constraints of player 1 (Eq. (1)). Since there must be no incentive to deviate from to , it holds
Similar conditions also hold for the deviation from to and that from to . Thus, we can write:
As a result, we can conclude that, if only recommends player 2 to play , , and , then . However, if player 2 decides to OptOut, then she would get at least , as and player 2 is guaranteed to get by playing . Thus, being stable, it must be the case that player 2 is always recommended to play in . Thus, , which is a contradiction. ∎
Finally, we prove that the stable distributions for the SG without followers encompass those defining CCEs.
Theorem 10.
.
Proof.
Let be a CCE of a given finite game . We prove that the SG admits a stable distribution with . In order to do so, for every leader , we let be such that , as shown in the following. Let us fix a player and let be such that, for every :
(3) 
i.e., is the best player ’s strategy against the correlated distribution . Given that