1 Introduction
Stackelberg equilibrium problems are very popular within the systemandcontrol community, since they offer a multiagent, decisionmaking framework that enables to model not only “horizontal” but also “vertical” interdependent relationships among heterogeneous agents, which are therefore clustered into leaders and followers. The application domains of Stackelberg equilibrium problems are, indeed, numerous, spanning from wireless networks, telecommunications [liu2019sinr], and network security [TongwenChen2018], to demand response and energy management [motalleb2019networked, chen2017stackelberg, mendoza2019online], economics [hirose2019comparing], and traffic control [groot2017hierarchical].
In its most general setting, a Stackelberg equilibrium problem between a leader and a set of followers can be formulated as a MPEC [luo1996mathematical, §1.2] or, in some specific cases, as an MPCC [scheel2000mathematical]. Both MPEC and MPCC are usually challenging to solve. Specifically, they are inherently illposed, nonconvex optimization problems, since typically there are no feasible solutions strictly lying in the interior of the feasible set, which may even be disconnected, implying that any constraint qualification is violated at every feasible point [jongen1991nonlinear]. It follows that, in this context, the basic convergence assumptions characterizing standard constrained optimization algorithms are not satisfied. Therefore, available solution methods are either tailored to the specific problem considered, or designed ad hoc for a subclass of MPEC/MPCC.
Algorithmic solution techniques for the class of games involving dominant and nondominant strategies, i.e. leaders and followers, trace back to the 70s. For example, openloop and feedback control policies for differential, hence continuoustime, unconstrained games were designed in [simaan1973stackelberg, kydland1977equilibrium], while in [kydland1975noncooperative] a comparison between finite/infinite horizon control strategies involving discretetime dynamics was proposed. More recently, a singleleader, multifollower differential game, modeling a pricing scheme for the Internet by basing on the bandwidth usage of the users, i.e., with congestion constraints, was solved in [bacsar2002stackelberg], and an iterative procedure to compute a StackelbergNashsaddle point for an unconstrained, singleleader, multifollower game with discretetime dynamics was proposed in [kebriaei2017discrete]. By relying on the uniqueness of the followers’ equilibrium for each leader’s strategy, standard fixedpoint algorithms are also proposed in [tushar2012economics, zou2017decentralized]. A first attempt to solve an MPEC modelling a more elaborated multileader, multifollower game, was investigated in [kulkarni2015existence]
. Specifically, the authors established the equivalence to a singleleader, multifollower game whenever the cost functions of the leaders admit a potential function and, in addition, the set of leaders has an identical conjecture or estimate on the follower equilibrium. Similar arguments are also exploited in
[leyffer2010solving] to address the same multileader, multifollower equilibrium problem. In this latter case, for each leader, the authors proposed a singleleader, multifollower game modelled as an MPEC. On the other hand, all these subgames, which are parametric in the decisions of the followers, are coupled together through a game against the leaders themselves. However, in both papers the solution to the singleleader, multifollower game remains to be dealt with, mainly due to the presence of nonconvexities and equilibrium/complementarity constraints which characterize MPEC/MPCC. Early algorithmic works on MPCC to solve singleleader, multifollower Stackelberg games, such as GaussSeidel or Jacobi [hobbs2001linear, ehrenmann2009comparison], are computationally expensive, especially for large number of followers. Additionally, they introduce several privacy issues, since they are designed by relying on diagonalization techniques. In [su2004sequential], after relaxing the complementarity conditions, a solution to an MPCC is computed through nonlinear complementarity problems, towards driving the relaxation parameter to zero.Our work aims at filling the apparent lack in the aforementioned literature of scalable and privacy preserving solution algorithms for equilibrium problems with nonconvex data and complementarity conditions, i.e., MPEC/MPCC. Specifically, we leverage on the SCA to design a twolayer, semidecentralized algorithm suitable to iteratively compute a local solution to the Stackelberg equilibrium problem involving a single leader and multiple followers in aggregative form with coupling constraints. The main contributions of the paper are summarized as follows:

We reformulate the Stackelberg game as an MPCC by embedding it into the leader nonconvex optimization problem the equivalent KKT conditions to compute a vGNE [facchinei2007finite] for the followers’ game (§II);

We exploit a key result provided in [scholtes2001convergence] to locally relax the complementarity constraints, obtaining the MPCCLICQ [fletcher2006local, Def. 3.1], i.e., the LICQ of all the points inside a certain neighborhood of the originally formulated MPCC (§III);

Along the same lines of [scutari2014decomposition, scutari2017parallel], we propose to convexify the relaxed MPCC at every iteration of the outer loop, whose optimal solution, computed within the inner loop, points a descent direction for the cost function of the original MPCC. By pursuing such a descent direction, the sequence of feasible points generated by the outer loop directly leads to a local solution of the Stackelberg equilibrium problem (§III);

We analyze the performance of the proposed algorithm applied to a numerical instance of the charging coordination problem for a fleet of PEV, also investigating the behavior of the leader and the followers as the regularization parameter varies (§IV).
To the best of our knowledge, the proposed twolayer algorithm represents the first attempt to compute a local solution to the Stackelberg equilibrium problem involving nonconvex data and equilibrium constraints by directly exploiting (and preserving) the hierarchical, multiagent structure of the original aggregative game.
Notation
, and denote the set of natural, real and nonnegative real numbers.
represents a vector with all elements equal to
. For vectors and , we denote and . We also use . means that and are orthogonal vectors. Given a matrix , denotes its transpose. represents the Kronecker product between the matrices and . For a function , denotes the approximation of at some . For a setvalued mapping , denotes its graph.2 Mathematical setup
2.1 Stackelberg game
We consider a hierarchical noncooperative game with one leader, controlling its decision variable , and followers, indexed by the set , where each follower controls its own variable , , , and aims at solving the following optimization problem:
(1) 
for some cost function . Let , , be the collective vector of strategies of the followers, while stacks all the local decision variables except the th one. We postulate the following standard assumptions on the followers’ data in (1).
Standing Assumption 1
For each , the function is convex and continuously differentiable, for fixed .
Standing Assumption 2
For each , .
In (1), each matrix stacks linear coupling constraints, while is the vector of shared resources among the followers. Let . Then, we preliminary define the sets and .
For a fixed strategy of the leader, , the followers aim to solve a GNEP. Specifically, by focusing on vGNE, such problem is equivalent to solve VI [facchinei2007finite], where, in view of Standing Assumption 1, is a continuously differentiable setvalued mapping defined as . This fact, along with the properties of , guarantee the nonemptiness of the set of vGNE that, for any , corresponds to the set
(2) 
On the other hand, the optimization problem of the leader reads as:
(3) 
for some cost function and local constraint set characterized by the following standard conditions.
Standing Assumption 3
The set is nonempty, closed and convex.
Standing Assumption 4
The function is coercive, its gradient is Lipschitz continuous on with constant .
We note that (3) defines an MPEC where is not strictly within the leader’s control, but it corresponds to an optimistic conjecture [kulkarni2015existence]. In view of [luo1996mathematical, Th. 1.4.1], the MPEC in (3) admits an optimal solution, since the coerciveness of implies compactness of its level sets, and the feasible set, , is closed under the postulated assumptions. Therefore, this ensures existence of a solution to the hierarchical game, according to the following notion of local generalized Stackelberg equilibrium, inspired by [hu2007using, kulkarni2015existence].
Definition 1
Informally speaking, at an lSE, the leader and the followers locally fulfill the set of mutually coupling constraints and none of them can gain by unilaterally deviating from their current strategy. Note that we refer to an SE if Definition 1 holds true with , i.e., and , thus coinciding with [kulkarni2015existence, Def. 1.1].
2.2 Aggregative game formulation
For computational purposes, we consider the cost function of the followers and leader to be in aggregative form, i.e.,
(4)  
where , , and . In view of Standing Assumption 1, given any feasible , it follows from [facchinei2007gen, Th. 3.1] that a set of strategies is a vGNE of the followers game in (1) if and only if the following coupled KKT conditions hold true:
which, in our aggregative setup, can be compactly rewritten as
(5) 
where , is the dual variable associated with , is the (local) dual variable associated with the local constraints defining , , and
Finally, by substituting back the KKT conditions in (5) into the optimization problem of the leader in (3), the problem of finding an SE of the hierarchical game in (1)–(3) can be equivalently written as
(6) 
2.3 Complementarity constraints relaxation
We note that the leader nonconvex optimization problem in (6) is an MPCC and, in general, it does not satisfy any standard constraint qualification. Therefore, we propose to study a regularized version by introducing slack variables and , , together with parameters , , which enable us to replace the complementarity constraints in (6) with the nonlinear constraints and , for all [scholtes2001convergence]. Thus, after defining , , , the regularized version of (6) reads as:
(7) 
where , , , and
For any given , , let us now introduce the sets
(8)  
Here, each and , , is a symmetric matrix with identities of suitable dimension on the antidiagonal. Furthermore, we define , where for brevity we omit the dependency from , explicated in . Finally, by introducing and , the closed, nonconvex feasible set of in (7) reads as
(9) 
We recall now the notion of MPCCLICQ for the MPCC in (6), which is characterized by the result stated immediately below.
Lemma 1
([scholtes2001convergence, Lemma 2.1]) Let . If satisfies the MPCCLICQ for the MPCC in (6), then there exists an open neighborhood of and scalars , , for all , such that, for every and , for all , the LICQ holds true at every point of .
Then, let us introduce the following fundamental assumption.
Standing Assumption 5
There exists some that satisfies the MPCCLICQ for the MPCC in (6). The regularization parameters are chosen so that and , for all .
In view of Standing Assumption 5, there exists a neighborhood such that locally satisfies the LICQ. As shown in §4.2, the coefficients , , , play a tradeoff role between the distance from a vGNE for the followers and a lower cost for the leader. To conclude the section, we stress that an optimal solution to (7), whose existence follows by its local LICQ and the coerciveness of , generates a pair that corresponds to an lSE of the original hierarchical game in (1)–(3).
3 Local Stackelberg equilibrium seeking
via sequential convex approximation
3.1 A twolayer algorithm
In the spirit of [scutari2014decomposition, scutari2017parallel], we then investigate how to solve (7) in a decentralized fashion by means of a twolayer algorithm, while preserving the hierarchical structure of the game (1)–(3). First, we linearize the nonlinear terms appearing in the cost function around some . Specifically, with , is linearized by following a first order Taylor expansion as where, for our aggregative game, we have:
According to [scutari2017parallel, §III.A], for the nonlinear constraints defining the sets in (8), we compute an upper approximation by observing that, e.g., . Thus, after linearizing the concave term around some , we define
The same procedure can be applied to each to obtain . Accordingly, is approximated by , with , while by
(10) 
Finally, by discarding constant terms and introducing , the convexified version of in (7) reads as
(11) 
where we add a “proximallike” term to the linearized cost function in (7) with . Hence, the cost function in (11), namely , is characterized as follows.
Lemma 2
The following statements hold true:

Given any , is uniformly strongly convex on , , with coefficient ;

Given any , is uniformly Lipschitz continuous on with coefficient .
(i) The statement directly follows by applying the definition of uniform strong convexity on the set .
(ii) Let . For any given , we have:
Remark 1
According to the structure of the vector , the coefficient may be replaced with locally defined , , without affecting the results given in the remainder, see [scutari2014decomposition, §III.A]. For simplicity, we adopt a unique, globally known parameter .
Thus, given any , in (11) admits a unique optimal solution associated with the mapping , with , defined as follows:
(12) 
For computing an SE, we propose the iterative procedure summarized in Algorithm 1, which is composed of two main loops and resorts on the so called SCA method. Specifically, once fixed the coefficients , for all , at each iteration , the outer loop is in charge of providing a feasible set of strategies , which are used to convexify (S1). Then, after solving the inner loop by computing the optimal solution to (S2), the outer loop updates the strategies (S3) to find a new approximation , and the procedure repeats until a certain stopping criterion is met.
3.2 Convergence analysis
First, we characterize the sequence generated by Algorithm 1 in terms of iterate feasibility. Then, we establish a key property of the mapping , and finally we prove that converges to an optimal solution to (7), generating an lSE of the hierarchical aggregative game (1)–(3), according to Definition 1.
Lemma 3
The following inclusions hold true:

, for all ;

.
(i) The upper approximation of the nonlinear constraints, which holds true for all , implies and , . Therefore, , and in view of the definitions in (9) and (10), inclusion (i) can be deduced.
(ii) First, in view of the approximation of the constraints, note that , for all , with convex subset of . Then, the proof follows by induction by considering that is a convex combination of and .
Lemma 4
For every , the vector is a descent direction for in , evaluated at , i.e., .
Given any , by definition, satisfies the minimum principle for (11), i.e., for all . From Lemma 3(ii), we choose , and by adding and subtracting the term , we obtain
By directly replacing with , the term on the lefthand side is equal to , while the one on the righthand side, in view of Lemma 2(i), is bounded from below by , leading to
Before establishing the convergence to an SE for the sequence generated by Algorithm 1, we recall a key result provided in [scutari2017parallel].
Lemma 5
Theorem 1
By combining the descent lemma [bertsekas1997nonlinear, Prop. A.24] and Lemma 4, the step (S3) in Algorithm 1 leads to:
where the second inequality follows from . If , then shall converge to a finite value, since can not happen in view of Standing Assumption 4. Thus, the convergence of implies , and therefore the bounded sequence in view of Lemma 3, and has a limit point in . From Lemma 5, such a limit point is a stationary solution to , and since is a strictly decreasing sequence, no limit point can be a local maximum of . Thus, converges to an optimal solution to (7), which subvector is an lSE of the original hierarchical game in (1)–(3).
Remark 2
If the parameters and are not globally known, Theorem 1 can be equivalently restated according to a vanishing stepsize rule, i.e., that shall be chosen so that , for all , and .
3.3 An augmented Lagrangian approach to solve the inner loop
A scalable and privacypreserving algorithm, suitable to solve (S2) in Algorithm 1 by exploiting the hierarchical structure of the original game, is the ADAL method proposed in [chatzipanagiotis2017convergence]. Since we are interested in finding the optimal solution to , from now on we omit the dependence on (unless differently specified) to alleviate the notation.

Leader:
Followers:
Coordinator:
Thus, at every iteration of the outer loop, the Lagrangian function associated to (11) is defined as
(13) 
where , and , , is the dual variable associated with the linear equality constraints. Note that the Lagrangian in (13) can be rewritten as the sum of terms associated to different entities, which happens to correspond to leader, the set of followers, and a central coordinator, respectively. In details, we define , , and . In light of [chatzipanagiotis2017convergence], we augment each one of these terms as, e.g., ( and are identical), where is a penalty term to be designed freely.
The main steps of the proposed semidecentralized procedure are summarized in Algorithm 2, where we emphasize that each augmented Lagrangian term depends on the linearization at the current outer iteration . Specifically, at every iteration of the inner loop, the ADAL requires that the followers, the leader and the central coordinator compute in parallel a minimization step of the local augmented Lagrangian. Here, , and are auxiliary variables introduced for privacy purposes and, given some , are locally updated. Finally, the central coordinator, which in some practical applications may eventually coincide with the leader, gathers and from the leader and followers, and updates the dual variable.
Proposition 1
The proof follows by noticing that satisfies the assumptions in [chatzipanagiotis2017convergence, Th. 2], for all . Specifically, is a closed and convex set, is infcompact and each one of its terms is twice continuously differentiable. Finally, Lemma 1 provides the local LICQ for , directly inherited by .
Comments
There are no comments yet.