Stackelberg equilibrium problems are very popular within the system-and-control community, since they offer a multi-agent, decision-making framework that enables to model not only “horizontal” but also “vertical” interdependent relationships among heterogeneous agents, which are therefore clustered into leaders and followers. The application domains of Stackelberg equilibrium problems are, indeed, numerous, spanning from wireless networks, telecommunications [liu2019sinr], and network security [TongwenChen2018], to demand response and energy management [motalleb2019networked, chen2017stackelberg, mendoza2019online], economics [hirose2019comparing], and traffic control [groot2017hierarchical].
In its most general setting, a Stackelberg equilibrium problem between a leader and a set of followers can be formulated as a MPEC [luo1996mathematical, §1.2] or, in some specific cases, as an MPCC [scheel2000mathematical]. Both MPEC and MPCC are usually challenging to solve. Specifically, they are inherently ill-posed, nonconvex optimization problems, since typically there are no feasible solutions strictly lying in the interior of the feasible set, which may even be disconnected, implying that any constraint qualification is violated at every feasible point [jongen1991nonlinear]. It follows that, in this context, the basic convergence assumptions characterizing standard constrained optimization algorithms are not satisfied. Therefore, available solution methods are either tailored to the specific problem considered, or designed ad hoc for a sub-class of MPEC/MPCC.
Algorithmic solution techniques for the class of games involving dominant and nondominant strategies, i.e. leaders and followers, trace back to the 70s. For example, open-loop and feedback control policies for differential, hence continuous-time, unconstrained games were designed in [simaan1973stackelberg, kydland1977equilibrium], while in [kydland1975noncooperative] a comparison between finite/infinite horizon control strategies involving discrete-time dynamics was proposed. More recently, a single-leader, multi-follower differential game, modeling a pricing scheme for the Internet by basing on the bandwidth usage of the users, i.e., with congestion constraints, was solved in [bacsar2002stackelberg], and an iterative procedure to compute a Stackelberg-Nash-saddle point for an unconstrained, single-leader, multi-follower game with discrete-time dynamics was proposed in [kebriaei2017discrete]. By relying on the uniqueness of the followers’ equilibrium for each leader’s strategy, standard fixed-point algorithms are also proposed in [tushar2012economics, zou2017decentralized]. A first attempt to solve an MPEC modelling a more elaborated multi-leader, multi-follower game, was investigated in [kulkarni2015existence]
. Specifically, the authors established the equivalence to a single-leader, multi-follower game whenever the cost functions of the leaders admit a potential function and, in addition, the set of leaders has an identical conjecture or estimate on the follower equilibrium. Similar arguments are also exploited in[leyffer2010solving] to address the same multi-leader, multi-follower equilibrium problem. In this latter case, for each leader, the authors proposed a single-leader, multi-follower game modelled as an MPEC. On the other hand, all these sub-games, which are parametric in the decisions of the followers, are coupled together through a game against the leaders themselves. However, in both papers the solution to the single-leader, multi-follower game remains to be dealt with, mainly due to the presence of nonconvexities and equilibrium/complementarity constraints which characterize MPEC/MPCC. Early algorithmic works on MPCC to solve single-leader, multi-follower Stackelberg games, such as Gauss-Seidel or Jacobi [hobbs2001linear, ehrenmann2009comparison], are computationally expensive, especially for large number of followers. Additionally, they introduce several privacy issues, since they are designed by relying on diagonalization techniques. In [su2004sequential], after relaxing the complementarity conditions, a solution to an MPCC is computed through nonlinear complementarity problems, towards driving the relaxation parameter to zero.
Our work aims at filling the apparent lack in the aforementioned literature of scalable and privacy preserving solution algorithms for equilibrium problems with nonconvex data and complementarity conditions, i.e., MPEC/MPCC. Specifically, we leverage on the SCA to design a two-layer, semi-decentralized algorithm suitable to iteratively compute a local solution to the Stackelberg equilibrium problem involving a single leader and multiple followers in aggregative form with coupling constraints. The main contributions of the paper are summarized as follows:
We reformulate the Stackelberg game as an MPCC by embedding it into the leader nonconvex optimization problem the equivalent KKT conditions to compute a v-GNE [facchinei2007finite] for the followers’ game (§II);
We exploit a key result provided in [scholtes2001convergence] to locally relax the complementarity constraints, obtaining the MPCC-LICQ [fletcher2006local, Def. 3.1], i.e., the LICQ of all the points inside a certain neighborhood of the originally formulated MPCC (§III);
Along the same lines of [scutari2014decomposition, scutari2017parallel], we propose to convexify the relaxed MPCC at every iteration of the outer loop, whose optimal solution, computed within the inner loop, points a descent direction for the cost function of the original MPCC. By pursuing such a descent direction, the sequence of feasible points generated by the outer loop directly leads to a local solution of the Stackelberg equilibrium problem (§III);
We analyze the performance of the proposed algorithm applied to a numerical instance of the charging coordination problem for a fleet of PEV, also investigating the behavior of the leader and the followers as the regularization parameter varies (§IV).
To the best of our knowledge, the proposed two-layer algorithm represents the first attempt to compute a local solution to the Stackelberg equilibrium problem involving nonconvex data and equilibrium constraints by directly exploiting (and preserving) the hierarchical, multi-agent structure of the original aggregative game.
, and denote the set of natural, real and nonnegative real numbers.
represents a vector with all elements equal to. For vectors and , we denote and . We also use . means that and are orthogonal vectors. Given a matrix , denotes its transpose. represents the Kronecker product between the matrices and . For a function , denotes the approximation of at some . For a set-valued mapping , denotes its graph.
2 Mathematical setup
2.1 Stackelberg game
We consider a hierarchical noncooperative game with one leader, controlling its decision variable , and followers, indexed by the set , where each follower controls its own variable , , , and aims at solving the following optimization problem:
for some cost function . Let , , be the collective vector of strategies of the followers, while stacks all the local decision variables except the -th one. We postulate the following standard assumptions on the followers’ data in (1).
Standing Assumption 1
For each , the function is convex and continuously differentiable, for fixed .
Standing Assumption 2
For each , .
In (1), each matrix stacks linear coupling constraints, while is the vector of shared resources among the followers. Let . Then, we preliminary define the sets and .
For a fixed strategy of the leader, , the followers aim to solve a GNEP. Specifically, by focusing on v-GNE, such problem is equivalent to solve VI [facchinei2007finite], where, in view of Standing Assumption 1, is a continuously differentiable set-valued mapping defined as . This fact, along with the properties of , guarantee the nonemptiness of the set of v-GNE that, for any , corresponds to the set
On the other hand, the optimization problem of the leader reads as:
for some cost function and local constraint set characterized by the following standard conditions.
Standing Assumption 3
The set is nonempty, closed and convex.
Standing Assumption 4
The function is coercive, its gradient is Lipschitz continuous on with constant .
We note that (3) defines an MPEC where is not strictly within the leader’s control, but it corresponds to an optimistic conjecture [kulkarni2015existence]. In view of [luo1996mathematical, Th. 1.4.1], the MPEC in (3) admits an optimal solution, since the coerciveness of implies compactness of its level sets, and the feasible set, , is closed under the postulated assumptions. Therefore, this ensures existence of a solution to the hierarchical game, according to the following notion of local generalized Stackelberg equilibrium, inspired by [hu2007using, kulkarni2015existence].
Informally speaking, at an l-SE, the leader and the followers locally fulfill the set of mutually coupling constraints and none of them can gain by unilaterally deviating from their current strategy. Note that we refer to an SE if Definition 1 holds true with , i.e., and , thus coinciding with [kulkarni2015existence, Def. 1.1].
2.2 Aggregative game formulation
For computational purposes, we consider the cost function of the followers and leader to be in aggregative form, i.e.,
where , , and . In view of Standing Assumption 1, given any feasible , it follows from [facchinei2007gen, Th. 3.1] that a set of strategies is a v-GNE of the followers game in (1) if and only if the following coupled KKT conditions hold true:
which, in our aggregative setup, can be compactly rewritten as
where , is the dual variable associated with , is the (local) dual variable associated with the local constraints defining , , and
Finally, by substituting back the KKT conditions in (5) into the optimization problem of the leader in (3), the problem of finding an SE of the hierarchical game in (1)–(3) can be equivalently written as
2.3 Complementarity constraints relaxation
We note that the leader nonconvex optimization problem in (6) is an MPCC and, in general, it does not satisfy any standard constraint qualification. Therefore, we propose to study a regularized version by introducing slack variables and , , together with parameters , , which enable us to replace the complementarity constraints in (6) with the nonlinear constraints and , for all [scholtes2001convergence]. Thus, after defining , , , the regularized version of (6) reads as:
where , , , and
For any given , , let us now introduce the sets
Here, each and , , is a symmetric matrix with identities of suitable dimension on the anti-diagonal. Furthermore, we define , where for brevity we omit the dependency from , explicated in . Finally, by introducing and , the closed, nonconvex feasible set of in (7) reads as
We recall now the notion of MPCC-LICQ for the MPCC in (6), which is characterized by the result stated immediately below.
([scholtes2001convergence, Lemma 2.1]) Let . If satisfies the MPCC-LICQ for the MPCC in (6), then there exists an open neighborhood of and scalars , , for all , such that, for every and , for all , the LICQ holds true at every point of .
Then, let us introduce the following fundamental assumption.
Standing Assumption 5
There exists some that satisfies the MPCC-LICQ for the MPCC in (6). The regularization parameters are chosen so that and , for all .
In view of Standing Assumption 5, there exists a neighborhood such that locally satisfies the LICQ. As shown in §4.2, the coefficients , , , play a trade-off role between the distance from a v-GNE for the followers and a lower cost for the leader. To conclude the section, we stress that an optimal solution to (7), whose existence follows by its local LICQ and the coerciveness of , generates a pair that corresponds to an l-SE of the original hierarchical game in (1)–(3).
3 Local Stackelberg equilibrium seeking
via sequential convex approximation
3.1 A two-layer algorithm
In the spirit of [scutari2014decomposition, scutari2017parallel], we then investigate how to solve (7) in a decentralized fashion by means of a two-layer algorithm, while preserving the hierarchical structure of the game (1)–(3). First, we linearize the nonlinear terms appearing in the cost function around some . Specifically, with , is linearized by following a first order Taylor expansion as where, for our aggregative game, we have:
According to [scutari2017parallel, §III.A], for the nonlinear constraints defining the sets in (8), we compute an upper approximation by observing that, e.g., . Thus, after linearizing the concave term around some , we define
The same procedure can be applied to each to obtain . Accordingly, is approximated by , with , while by
Finally, by discarding constant terms and introducing , the convexified version of in (7) reads as
The following statements hold true:
Given any , is uniformly strongly convex on , , with coefficient ;
Given any , is uniformly Lipschitz continuous on with coefficient .
(i) The statement directly follows by applying the definition of uniform strong convexity on the set .
(ii) Let . For any given , we have:
According to the structure of the vector , the coefficient may be replaced with locally defined , , without affecting the results given in the remainder, see [scutari2014decomposition, §III.A]. For simplicity, we adopt a unique, globally known parameter .
Thus, given any , in (11) admits a unique optimal solution associated with the mapping , with , defined as follows:
For computing an -SE, we propose the iterative procedure summarized in Algorithm 1, which is composed of two main loops and resorts on the so called SCA method. Specifically, once fixed the coefficients , for all , at each iteration , the outer loop is in charge of providing a feasible set of strategies , which are used to convexify (S1). Then, after solving the inner loop by computing the optimal solution to (S2), the outer loop updates the strategies (S3) to find a new approximation , and the procedure repeats until a certain stopping criterion is met.
3.2 Convergence analysis
First, we characterize the sequence generated by Algorithm 1 in terms of iterate feasibility. Then, we establish a key property of the mapping , and finally we prove that converges to an optimal solution to (7), generating an l-SE of the hierarchical aggregative game (1)–(3), according to Definition 1.
The following inclusions hold true:
, for all ;
(ii) First, in view of the approximation of the constraints, note that , for all , with convex subset of . Then, the proof follows by induction by considering that is a convex combination of and .
For every , the vector is a descent direction for in , evaluated at , i.e., .
By directly replacing with , the term on the left-hand side is equal to , while the one on the right-hand side, in view of Lemma 2(i), is bounded from below by , leading to
Before establishing the convergence to an -SE for the sequence generated by Algorithm 1, we recall a key result provided in [scutari2017parallel].
where the second inequality follows from . If , then shall converge to a finite value, since can not happen in view of Standing Assumption 4. Thus, the convergence of implies , and therefore the bounded sequence in view of Lemma 3, and has a limit point in . From Lemma 5, such a limit point is a stationary solution to , and since is a strictly decreasing sequence, no limit point can be a local maximum of . Thus, converges to an optimal solution to (7), which subvector is an l-SE of the original hierarchical game in (1)–(3).
If the parameters and are not globally known, Theorem 1 can be equivalently restated according to a vanishing step-size rule, i.e., that shall be chosen so that , for all , and .
3.3 An augmented Lagrangian approach to solve the inner loop
A scalable and privacy-preserving algorithm, suitable to solve (S2) in Algorithm 1 by exploiting the hierarchical structure of the original game, is the ADAL method proposed in [chatzipanagiotis2017convergence]. Since we are interested in finding the optimal solution to , from now on we omit the dependence on (unless differently specified) to alleviate the notation.
Thus, at every iteration of the outer loop, the Lagrangian function associated to (11) is defined as
where , and , , is the dual variable associated with the linear equality constraints. Note that the Lagrangian in (13) can be rewritten as the sum of terms associated to different entities, which happens to correspond to leader, the set of followers, and a central coordinator, respectively. In details, we define , , and . In light of [chatzipanagiotis2017convergence], we augment each one of these terms as, e.g., ( and are identical), where is a penalty term to be designed freely.
The main steps of the proposed semi-decentralized procedure are summarized in Algorithm 2, where we emphasize that each augmented Lagrangian term depends on the linearization at the current outer iteration . Specifically, at every iteration of the inner loop, the ADAL requires that the followers, the leader and the central coordinator compute in parallel a minimization step of the local augmented Lagrangian. Here, , and are auxiliary variables introduced for privacy purposes and, given some , are locally updated. Finally, the central coordinator, which in some practical applications may eventually coincide with the leader, gathers and from the leader and followers, and updates the dual variable.
The proof follows by noticing that satisfies the assumptions in [chatzipanagiotis2017convergence, Th. 2], for all . Specifically, is a closed and convex set, is inf-compact and each one of its terms is twice continuously differentiable. Finally, Lemma 1 provides the local LICQ for , directly inherited by .