# Local Stackelberg equilibrium seeking in generalized aggregative games

We propose a two-layer, semi-decentralized algorithm to compute a local solution to the Stackelberg equilibrium problem in aggregative games with coupling constraints. Specifically, we focus on a single-leader, multiple-follower problem, and after equivalently recasting the Stackelberg game as a mathematical program with complementarity constraints (MPCC), we iteratively convexify a regularized version of the MPCC as inner problem, whose solution generates a sequence of feasible descent directions for the original MPCC. Thus, by pursuing a descent direction at every outer iteration, we establish convergence to a local Stackelberg equilibrium. Finally, the proposed algorithm is tested on a numerical case study involving a hierarchical instance of the charging coordination of Plug-in Electric Vehicles (PEVs).

## Authors

• 9 publications
• 1 publication
• 2 publications
• 34 publications
• ### A Douglas-Rachford splitting for semi-decentralized generalized Nash equilibrium seeking in Monotone Aggregative Games

We address the generalized Nash equilibrium seeking problem for noncoope...
03/28/2018 ∙ by Giuseppe Belgioioso, et al. ∙ 0

• ### Continuous-time integral dynamics for monotone aggregative games with coupling constraints

We consider continuous-time equilibrium seeking in monotone aggregative ...
05/08/2018 ∙ by Claudio De Persis, et al. ∙ 0

• ### Charging plug-in electric vehicles as a mixed-integer aggregative game

We consider the charge scheduling coordination of a fleet of plug-in ele...
10/22/2019 ∙ by Carlo Cenedese, et al. ∙ 0

• ### Coordinating Followers to Reach Better Equilibria: End-to-End Gradient Descent for Stackelberg Games

A growing body of work in game theory extends the traditional Stackelber...
06/06/2021 ∙ by Kai Wang, et al. ∙ 0

• ### Continuous-time integral dynamics for Aggregative Game equilibrium seeking

In this paper, we consider continuous-time semi-decentralized dynamics f...
03/28/2018 ∙ by Claudio De Persis, et al. ∙ 0

• ### Time-varying constrained proximal type dynamics in multi-agent network games

In this paper, we study multi-agent network games subject to affine time...
11/19/2019 ∙ by Carlo Cenedese, et al. ∙ 0

• ### Computing a Pessimistic Leader-Follower Equilibrium with Multiple Followers: the Mixed-Pure Case

The search problem of computing a leader-follower equilibrium has been w...
08/04/2018 ∙ by Stefano Coniglio, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Stackelberg equilibrium problems are very popular within the system-and-control community, since they offer a multi-agent, decision-making framework that enables to model not only “horizontal” but also “vertical” interdependent relationships among heterogeneous agents, which are therefore clustered into leaders and followers. The application domains of Stackelberg equilibrium problems are, indeed, numerous, spanning from wireless networks, telecommunications [liu2019sinr], and network security [TongwenChen2018], to demand response and energy management [motalleb2019networked, chen2017stackelberg, mendoza2019online], economics [hirose2019comparing], and traffic control [groot2017hierarchical].

In its most general setting, a Stackelberg equilibrium problem between a leader and a set of followers can be formulated as a MPEC [luo1996mathematical, §1.2] or, in some specific cases, as an MPCC [scheel2000mathematical]. Both MPEC and MPCC are usually challenging to solve. Specifically, they are inherently ill-posed, nonconvex optimization problems, since typically there are no feasible solutions strictly lying in the interior of the feasible set, which may even be disconnected, implying that any constraint qualification is violated at every feasible point [jongen1991nonlinear]. It follows that, in this context, the basic convergence assumptions characterizing standard constrained optimization algorithms are not satisfied. Therefore, available solution methods are either tailored to the specific problem considered, or designed ad hoc for a sub-class of MPEC/MPCC.

Algorithmic solution techniques for the class of games involving dominant and nondominant strategies, i.e. leaders and followers, trace back to the 70s. For example, open-loop and feedback control policies for differential, hence continuous-time, unconstrained games were designed in [simaan1973stackelberg, kydland1977equilibrium], while in [kydland1975noncooperative] a comparison between finite/infinite horizon control strategies involving discrete-time dynamics was proposed. More recently, a single-leader, multi-follower differential game, modeling a pricing scheme for the Internet by basing on the bandwidth usage of the users, i.e., with congestion constraints, was solved in [bacsar2002stackelberg], and an iterative procedure to compute a Stackelberg-Nash-saddle point for an unconstrained, single-leader, multi-follower game with discrete-time dynamics was proposed in [kebriaei2017discrete]. By relying on the uniqueness of the followers’ equilibrium for each leader’s strategy, standard fixed-point algorithms are also proposed in [tushar2012economics, zou2017decentralized]. A first attempt to solve an MPEC modelling a more elaborated multi-leader, multi-follower game, was investigated in [kulkarni2015existence]

. Specifically, the authors established the equivalence to a single-leader, multi-follower game whenever the cost functions of the leaders admit a potential function and, in addition, the set of leaders has an identical conjecture or estimate on the follower equilibrium. Similar arguments are also exploited in

[leyffer2010solving] to address the same multi-leader, multi-follower equilibrium problem. In this latter case, for each leader, the authors proposed a single-leader, multi-follower game modelled as an MPEC. On the other hand, all these sub-games, which are parametric in the decisions of the followers, are coupled together through a game against the leaders themselves. However, in both papers the solution to the single-leader, multi-follower game remains to be dealt with, mainly due to the presence of nonconvexities and equilibrium/complementarity constraints which characterize MPEC/MPCC. Early algorithmic works on MPCC to solve single-leader, multi-follower Stackelberg games, such as Gauss-Seidel or Jacobi [hobbs2001linear, ehrenmann2009comparison], are computationally expensive, especially for large number of followers. Additionally, they introduce several privacy issues, since they are designed by relying on diagonalization techniques. In [su2004sequential], after relaxing the complementarity conditions, a solution to an MPCC is computed through nonlinear complementarity problems, towards driving the relaxation parameter to zero.

Our work aims at filling the apparent lack in the aforementioned literature of scalable and privacy preserving solution algorithms for equilibrium problems with nonconvex data and complementarity conditions, i.e., MPEC/MPCC. Specifically, we leverage on the SCA to design a two-layer, semi-decentralized algorithm suitable to iteratively compute a local solution to the Stackelberg equilibrium problem involving a single leader and multiple followers in aggregative form with coupling constraints. The main contributions of the paper are summarized as follows:

• We reformulate the Stackelberg game as an MPCC by embedding it into the leader nonconvex optimization problem the equivalent KKT conditions to compute a v-GNE [facchinei2007finite] for the followers’ game (§II);

• We exploit a key result provided in [scholtes2001convergence] to locally relax the complementarity constraints, obtaining the MPCC-LICQ [fletcher2006local, Def. 3.1], i.e., the LICQ of all the points inside a certain neighborhood of the originally formulated MPCC (§III);

• Along the same lines of [scutari2014decomposition, scutari2017parallel], we propose to convexify the relaxed MPCC at every iteration of the outer loop, whose optimal solution, computed within the inner loop, points a descent direction for the cost function of the original MPCC. By pursuing such a descent direction, the sequence of feasible points generated by the outer loop directly leads to a local solution of the Stackelberg equilibrium problem (§III);

• We analyze the performance of the proposed algorithm applied to a numerical instance of the charging coordination problem for a fleet of PEV, also investigating the behavior of the leader and the followers as the regularization parameter varies (§IV).

To the best of our knowledge, the proposed two-layer algorithm represents the first attempt to compute a local solution to the Stackelberg equilibrium problem involving nonconvex data and equilibrium constraints by directly exploiting (and preserving) the hierarchical, multi-agent structure of the original aggregative game.

### Notation

, and denote the set of natural, real and nonnegative real numbers.

represents a vector with all elements equal to

. For vectors and , we denote and . We also use . means that and are orthogonal vectors. Given a matrix , denotes its transpose. represents the Kronecker product between the matrices and . For a function , denotes the approximation of at some . For a set-valued mapping , denotes its graph.

## 2 Mathematical setup

### 2.1 Stackelberg game

We consider a hierarchical noncooperative game with one leader, controlling its decision variable , and followers, indexed by the set , where each follower controls its own variable , , , and aims at solving the following optimization problem:

 ∀i∈I:⎧⎨⎩minxi∈XiJi(y0,xi,x−i) s.t. Aixi+∑j∈I∖{i}Ajxj≤b, (1)

for some cost function . Let , , be the collective vector of strategies of the followers, while stacks all the local decision variables except the -th one. We postulate the following standard assumptions on the followers’ data in (1).

###### Standing Assumption 1

For each , the function is convex and continuously differentiable, for fixed .

###### Standing Assumption 2

For each , .

In (1), each matrix stacks linear coupling constraints, while is the vector of shared resources among the followers. Let . Then, we preliminary define the sets and .

For a fixed strategy of the leader, , the followers aim to solve a GNEP. Specifically, by focusing on v-GNE, such problem is equivalent to solve VI [facchinei2007finite], where, in view of Standing Assumption 1, is a continuously differentiable set-valued mapping defined as . This fact, along with the properties of , guarantee the nonemptiness of the set of v-GNE that, for any , corresponds to the set

 S(y0)\coloneqq{x∈Θ∣(z−x)⊤H(y0,x)≥0,∀z∈Θ}. (2)

On the other hand, the optimization problem of the leader reads as:

 {% miny0,xJ0(y0,x) s.t. (y0,x)∈% gph(S)∩(Y0×Rn), (3)

for some cost function and local constraint set characterized by the following standard conditions.

###### Standing Assumption 3

The set is nonempty, closed and convex.

###### Standing Assumption 4

The function is coercive, its gradient is Lipschitz continuous on with constant .

We note that (3) defines an MPEC where is not strictly within the leader’s control, but it corresponds to an optimistic conjecture [kulkarni2015existence]. In view of [luo1996mathematical, Th. 1.4.1], the MPEC in (3) admits an optimal solution, since the coerciveness of implies compactness of its level sets, and the feasible set, , is closed under the postulated assumptions. Therefore, this ensures existence of a solution to the hierarchical game, according to the following notion of local generalized Stackelberg equilibrium, inspired by [hu2007using, kulkarni2015existence].

###### Definition 1

A pair , with as in (2), is a l-SE of the hierarchical game in (1)–(3) if there exist open neighborhoods and of and , respectively, such that

 J0(y∗0,x∗)≤inf(y0,x)∈gph(S)∩OJ0(y0,x),

where .

Informally speaking, at an l-SE, the leader and the followers locally fulfill the set of mutually coupling constraints and none of them can gain by unilaterally deviating from their current strategy. Note that we refer to an SE if Definition 1 holds true with , i.e., and , thus coinciding with [kulkarni2015existence, Def. 1.1].

### 2.2 Aggregative game formulation

For computational purposes, we consider the cost function of the followers and leader to be in aggregative form, i.e.,

 Ji\coloneqq12x⊤iQixi+(1N∑j∈ICi,jxj+Ci,0y0)⊤xi,∀i∈I, (4) J0\coloneqqf0(y0)+(∑i∈If0,i(xi))⊤y0,

where , , and . In view of Standing Assumption 1, given any feasible , it follows from [facchinei2007gen, Th. 3.1] that a set of strategies is a v-GNE of the followers game in (1) if and only if the following coupled KKT conditions hold true:

 ⎧⎪⎨⎪⎩∇xiJi(y0,xi,x−i)+A⊤iλ+F⊤iλi=0,∀i∈I,0≤λ⊥−(Ax−b)≥0,0≤λi⊥−(Fixi−gi)≥0,∀i∈I,

which, in our aggregative setup, can be compactly rewritten as

 ⎧⎪⎨⎪⎩Qx+Cy0+A⊤λ+F⊤λ=0,0≤λ⊥−(Ax−b)≥0,0≤λi⊥−(Fixi−gi)≥0,∀i∈I, (5)

where , is the dual variable associated with , is the (local) dual variable associated with the local constraints defining , , and

 Q\coloneqq⎡⎢ ⎢ ⎢ ⎢⎣Q1+1NC1,1⋯1NC1,N⋮⋱⋮1NCN,1⋯QN+1NCN,N⎤⎥ ⎥ ⎥ ⎥⎦,C\coloneqq⎡⎢ ⎢⎣C10⋮CN0⎤⎥ ⎥⎦.

Finally, by substituting back the KKT conditions in (5) into the optimization problem of the leader in (3), the problem of finding an SE of the hierarchical game in (1)–(3) can be equivalently written as

 ⎧⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪⎩miny0,x,λ,λJ0(y0,x) s.t. Qx+Cy0+A⊤λ+F⊤λ=0,0≤λi⊥−(Fixi−gi)≥0,∀i∈I,0≤λ⊥−(Ax−b)≥0,y0∈Y0. (6)

### 2.3 Complementarity constraints relaxation

We note that the leader nonconvex optimization problem in (6) is an MPCC and, in general, it does not satisfy any standard constraint qualification. Therefore, we propose to study a regularized version by introducing slack variables and , , together with parameters , , which enable us to replace the complementarity constraints in (6) with the nonlinear constraints and , for all [scholtes2001convergence]. Thus, after defining , , , the regularized version of (6) reads as:

 R(θ):⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩miny0,y,νJ0(y0,x) s.t. Afy+Aℓy0+Acν=d,λ⊤iμi≤θi,λi,μi≥0,∀i∈I,λ⊤μ≤θ,λ,μ≥0,y0∈Y0, (7)

where , , , and

 Af\coloneqq⎡⎢⎣Q{[F⊤i0]}i∈IA0F[0I]⊗1⎤⎥⎦,Ac\coloneqq⎡⎢⎣A⊤00I00⎤⎥⎦.

For any given , , let us now introduce the sets

 C(θ) \coloneqq{ν∈R2m≥0∣12ν⊤Pν≤θ}, (8) Ci(θi) \coloneqq{νi∈R2pi≥0∣12ν⊤iPiνi≤θi},∀i∈I.

Here, each and , , is a symmetric matrix with identities of suitable dimension on the anti-diagonal. Furthermore, we define , where for brevity we omit the dependency from , explicated in . Finally, by introducing and , the closed, nonconvex feasible set of in (7) reads as

 R(θ)\coloneqq{ω∈Ω(θ)∣Aωω−d=0}. (9)

We recall now the notion of MPCC-LICQ for the MPCC in (6), which is characterized by the result stated immediately below.

###### Definition 2

The MPCC in (6) satisfies the MPCC-LICQ at if in (7) satisfies the LICQ at .

###### Lemma 1

([scholtes2001convergence, Lemma 2.1]) Let . If satisfies the MPCC-LICQ for the MPCC in (6), then there exists an open neighborhood of and scalars , , for all , such that, for every and , for all , the LICQ holds true at every point of .

Then, let us introduce the following fundamental assumption.

###### Standing Assumption 5

There exists some that satisfies the MPCC-LICQ for the MPCC in (6). The regularization parameters are chosen so that and , for all .

In view of Standing Assumption 5, there exists a neighborhood such that locally satisfies the LICQ. As shown in §4.2, the coefficients , , , play a trade-off role between the distance from a v-GNE for the followers and a lower cost for the leader. To conclude the section, we stress that an optimal solution to (7), whose existence follows by its local LICQ and the coerciveness of , generates a pair that corresponds to an l-SE of the original hierarchical game in (1)–(3).

## 3 Local Stackelberg equilibrium seeking via sequential convex approximation

### 3.1 A two-layer algorithm

In the spirit of [scutari2014decomposition, scutari2017parallel], we then investigate how to solve (7) in a decentralized fashion by means of a two-layer algorithm, while preserving the hierarchical structure of the game (1)–(3). First, we linearize the nonlinear terms appearing in the cost function around some . Specifically, with , is linearized by following a first order Taylor expansion as where, for our aggregative game, we have:

 ∇J0(φ) =col(∇y0f0(y0)+∑j∈If0,j(xj),{∇xjf0,j(xj)⊤y0}j∈I) \eqqcoloncol(cℓ(y0,x),cf(y0,x)).

According to [scutari2017parallel, §III.A], for the nonlinear constraints defining the sets in (8), we compute an upper approximation by observing that, e.g., . Thus, after linearizing the concave term around some , we define

 ~C(θ;¯ω)\coloneqq{ν∈R2m≥0∣12(1⊤ν)⊤(1⊤ν)−¯ν⊤ν+12¯ν⊤¯ν≤θ}.

The same procedure can be applied to each to obtain . Accordingly, is approximated by , with , while by

 ~R(θ;¯ω)\coloneqq{ω∈~Ω(θ;¯ω)∣Aωω−d=0}. (10)

Finally, by discarding constant terms and introducing , the convexified version of in (7) reads as

 ~R(θ;¯ω):⎧⎪⎨⎪⎩minω∈~Ω(θ;¯ω)cω(¯ω)⊤ω+σ2∥ω−¯ω∥2 s.t. Aωω=d, (11)

where we add a “proximal-like” term to the linearized cost function in (7) with . Hence, the cost function in (11), namely , is characterized as follows.

###### Lemma 2

The following statements hold true:

• Given any , is uniformly strongly convex on , , with coefficient ;

• Given any , is uniformly Lipschitz continuous on with coefficient .

(i) The statement directly follows by applying the definition of uniform strong convexity on the set .

(ii) Let . For any given , we have:

 ∥∇~J0(ω;ω1) −∇~J0(ω;ω2)∥=∥cω(ω1)−cω(ω2)+σ(ω2−ω1)∥ ≤∥col(∇J0(φ1),0)−col(∇J0(φ2),0)∥+σ∥ω1−ω2∥ ≤(κ0+σ)∥ω1−ω2∥.
###### Remark 1

According to the structure of the vector , the coefficient may be replaced with locally defined , , without affecting the results given in the remainder, see [scutari2014decomposition, §III.A]. For simplicity, we adopt a unique, globally known parameter .

Thus, given any , in (11) admits a unique optimal solution associated with the mapping , with , defined as follows:

 ^ω(¯ω)\coloneqqargminω∈~R(θ;¯ω)~J0(ω;¯ω). (12)

For computing an -SE, we propose the iterative procedure summarized in Algorithm 1, which is composed of two main loops and resorts on the so called SCA method. Specifically, once fixed the coefficients , for all , at each iteration , the outer loop is in charge of providing a feasible set of strategies , which are used to convexify (S1). Then, after solving the inner loop by computing the optimal solution to (S2), the outer loop updates the strategies (S3) to find a new approximation , and the procedure repeats until a certain stopping criterion is met.

### 3.2 Convergence analysis

First, we characterize the sequence generated by Algorithm 1 in terms of iterate feasibility. Then, we establish a key property of the mapping , and finally we prove that converges to an optimal solution to (7), generating an l-SE of the hierarchical aggregative game (1)–(3), according to Definition 1.

###### Lemma 3

The following inclusions hold true:

• , for all ;

• .

(i) The upper approximation of the nonlinear constraints, which holds true for all , implies and , . Therefore, , and in view of the definitions in (9) and (10), inclusion (i) can be deduced.

(ii) First, in view of the approximation of the constraints, note that , for all , with convex subset of . Then, the proof follows by induction by considering that is a convex combination of and .

###### Lemma 4

For every , the vector is a descent direction for in , evaluated at , i.e., .

Given any , by definition, satisfies the minimum principle for (11), i.e., for all . From Lemma 3(ii), we choose , and by adding and subtracting the term , we obtain

 (¯ω−^ω(¯ω))⊤ ∇~J0(¯ω;¯ω)≥ (¯ω−^ω(¯ω))⊤(∇~J0(¯ω;¯ω)−∇~J0(^ω(¯ω);¯ω))

By directly replacing with , the term on the left-hand side is equal to , while the one on the right-hand side, in view of Lemma 2(i), is bounded from below by , leading to

 (¯φ−^φ(¯ω))⊤∇J0(¯φ)≥σ∥¯ω−^ω(¯ω)∥2.

Before establishing the convergence to an -SE for the sequence generated by Algorithm 1, we recall a key result provided in [scutari2017parallel].

###### Lemma 5

([scutari2017parallel, Th. 14]) Let be the sequence generated by Algorithm 1 and assume that . Then, every limit point of generated by Algorithm 1 is a stationary solution to .

###### Theorem 1

Let in Algorithm 1 be chosen so that . Then, the sequence generated by Algorithm 1 converges to an optimal solution to in (7), which subvector is an l-SE of the hierarchical game in (1)–(3).

By combining the descent lemma [bertsekas1997nonlinear, Prop. A.24] and Lemma 4, the step (S3) in Algorithm 1 leads to:

 J0(φk+1) ≤J0(φk)+α∇⊤J0(φk)(^φ(ωk)−φk) +α2κ02∥^φ(ωk)−φk∥2 ≤J0(φk)−α(σ−ακ02)∥^ω(ωk)−ωk∥2,

where the second inequality follows from . If , then shall converge to a finite value, since can not happen in view of Standing Assumption 4. Thus, the convergence of implies , and therefore the bounded sequence in view of Lemma 3, and has a limit point in . From Lemma 5, such a limit point is a stationary solution to , and since is a strictly decreasing sequence, no limit point can be a local maximum of . Thus, converges to an optimal solution to (7), which subvector is an l-SE of the original hierarchical game in (1)–(3).

###### Remark 2

If the parameters and are not globally known, Theorem 1 can be equivalently restated according to a vanishing step-size rule, i.e., that shall be chosen so that , for all , and .

### 3.3 An augmented Lagrangian approach to solve the inner loop

A scalable and privacy-preserving algorithm, suitable to solve (S2) in Algorithm 1 by exploiting the hierarchical structure of the original game, is the ADAL method proposed in [chatzipanagiotis2017convergence]. Since we are interested in finding the optimal solution to , from now on we omit the dependence on (unless differently specified) to alleviate the notation.

Thus, at every iteration of the outer loop, the Lagrangian function associated to (11) is defined as

 Lk(ω,ν)=(ckω)⊤ω+σ2∥ω−ωk∥2+η⊤(Aωω−d), (13)

where , and , , is the dual variable associated with the linear equality constraints. Note that the Lagrangian in (13) can be rewritten as the sum of terms associated to different entities, which happens to correspond to leader, the set of followers, and a central coordinator, respectively. In details, we define , , and . In light of [chatzipanagiotis2017convergence], we augment each one of these terms as, e.g., ( and are identical), where is a penalty term to be designed freely.

The main steps of the proposed semi-decentralized procedure are summarized in Algorithm 2, where we emphasize that each augmented Lagrangian term depends on the linearization at the current outer iteration . Specifically, at every iteration of the inner loop, the ADAL requires that the followers, the leader and the central coordinator compute in parallel a minimization step of the local augmented Lagrangian. Here, , and are auxiliary variables introduced for privacy purposes and, given some , are locally updated. Finally, the central coordinator, which in some practical applications may eventually coincide with the leader, gathers and from the leader and followers, and updates the dual variable.

###### Proposition 1

Let be sufficiently large and , where corresponds to the maximum degree among the constraints in (10). Then, the sequence generated by Algorithm 2 converges to the minimizer of , for all .

The proof follows by noticing that satisfies the assumptions in [chatzipanagiotis2017convergence, Th. 2], for all . Specifically, is a closed and convex set, is inf-compact and each one of its terms is twice continuously differentiable. Finally, Lemma 1 provides the local LICQ for , directly inherited by .

###### Remark 3

For simplicity, we adopt a common to update the auxiliary variables , and . In principle, each entity involved within the ADAL in Algorithm 2 can locally set its own step size according to the degree of each constraint in (11), see [chatzipanagiotis2017convergence, §II.A].