# Continuous-time fully distributed generalized Nash equilibrium seeking for multi-integrator agents

We consider a group of (multi)-integrator agents playing games on a network, in a partial-decision information scenario. We design fully distributed continuous-time controllers, based on consensus and primal-dual gradient dynamics, to drive the agents to a generalized Nash equilibrium. Our first solution adopts fixed gains, whose choice requires the knowledge of some global parameters of the game. Therefore, to adapt the procedure to setups where the agents do not have any global information, we introduce a controller that can be tuned in a completely decentralized fashion, thanks to the use of integral adaptive weights. We further introduce algorithms, both with constant and dynamic gains, specifically devised for generalized aggregative games. For all the proposed control schemes, we show convergence to a variational equilibrium, under Lipschitz continuity and strong monotonicity of the game mapping, by leveraging monotonicity properties and stability theory for projected dynamical systems.

## Authors

• 7 publications
• 35 publications
• ### A continuous-time distributed generalized Nash equilibrium seeking algorithm over networks for double-integrator agents

We consider a system of single- or double integrator agents playing a ge...
10/25/2019 ∙ by Mattia Bianchi, et al. ∙ 0

• ### A distributed proximal-point algorithm for Nash equilibrium seeking under partial-decision information with geometric convergence

We consider the Nash equilibrium seeking problem for a group of noncoope...
10/25/2019 ∙ by Mattia Bianchi, et al. ∙ 0

• ### Continuous-time integral dynamics for Aggregative Game equilibrium seeking

In this paper, we consider continuous-time semi-decentralized dynamics f...
03/28/2018 ∙ by Claudio De Persis, et al. ∙ 0

• ### Learning in Matrix Games can be Arbitrarily Complex

A growing number of machine learning architectures, such as Generative A...
03/05/2021 ∙ by Gabriel P. Andrade, et al. ∙ 0

• ### Fast generalized Nash equilibrium seeking under partial-decision information

We address the generalized Nash equilibrium (GNE) problem in a partial-d...
03/20/2020 ∙ by Mattia Bianchi, et al. ∙ 0

• ### Probably Approximately Correct Nash Equilibrium Learning

We consider a multi-agent noncooperative game with agents' objective fun...
03/25/2019 ∙ by Filiberto Fele, et al. ∙ 0

• ### Acquisition Games with Partial-Asymmetric Information

We consider an example of stochastic games with partial, asymmetric and ...
09/14/2019 ∙ by Veeraruna Kavitha, et al. ∙ 0

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

GNE problems arise in several engineering applications, including demand-side management in the smart grid [25], charging/discharging of electric vehicles [18], formation control [21], communication networks [14]. In these examples, multiple selfish decision makers, or agents, aim at optimizing their individual, yet inter-dependent, objective functions, subject to shared constraints. From a game-theoretic perspective, the goal is to design distributed GNE seeking algorithms, using the local information available to each agent. Moreover, in the cyber-physical sytems framework, games are often played by agents with their own dynamics [24], [28]. In this case, the “strategy” of each agent consists of the output of a dynamical system, and controllers have to be conceived to steer the physical processes to a Nash equilibrium, while ensuring closed-loop stability. Therefore, it is advantageous to consider continuous-time schemes, for which control-theoretic properties are more easily unraveled.
Literature review: A variety of different algorithms have been proposed to seek GNE in a distributed way [32], [33], [6]. A recent part of the literature focuses on aggregative games, for which the cost of each agent depends on the others agents’ strategy only via an aggregative function [2], [4], [7]. These works refers to (aggregative) games played in a full-information setting, where each agent can access the decision of all the competitors (aggregate value), for example in the presence of a central coordinator that broadcasts the data to the network. Nevertheless, in many applications, the existence of a node with bidirectional communication with all the agents may be impractical, and the agents can only rely on local information. One solution is offered by pay-off based methods [15], [28]

, that are decentralized, but require the agents to measure their cost functions. Alternatively, in this paper, we assume that the agents agree on sharing some information with their neighbors. Each agent keeps an estimate of all the competitors’ action and asymptotically reconstruct the true value, exploiting the data exchanged over the network. Such a partial-decision information scenario has been investigated for games without coupling constraints, resorting to (projected) gradient and consensus dynamics, both in discrete-time

[26], [20], and continuous-time [17], [30], [8]. Of major interest for this paper is the method in [8], where a nonlinear averaging integral controller is used to tune on-line the weights of the communication. The advantage is to guarantee convergence to a NE without requiring the knowledge of any global parameter or the use of a constant, high-enough, gain, which is the solution proposed in [17]. Fewer works deal with generalized games. A double-layer algorithm was presented in [31]. Remarkably, Pavel in [23] derived a single-timescale, fixed step-size GNE learning algorithm, by leveraging an elegant operator splitting approach. The authors of [11] addressed aggregative games with equality constraints, via continuous-time design. Moreover, all the results mentioned above consider static or single-integrator agents only. Distributively driving a network of more complex physical systems to game theoretic solutions is still a relatively unexplored problem. With regard to aggregative games, a proportional integral feedback algorithm was developed in [9] to seek a NE in networks of passive nonlinear second-order systems; in [10], [34], continuous-time gradient-based controllers were introduced for some classes of nonlinear dynamics with uncertainties. The authors of [28] addressed generally coupled cost games played by linear agents, via an extremum seeking approach. NE problems arising in systems of multi-integrator agents were studied in [24]. Moreover, all the references cited do not consider generalized games. Despite the scarcity of results, the presence of coupling constraints is a significant extension, that arises naturally in a variety of fields, when the agents share some common resource or limitation [12, §2].
Contributions: Motivated by the above, in this paper we investigate continuous-time GNE seeking for networks of multi-integrator agents. We consider games with affine coupling constraints, played under partial-decision information. Specifically:

• [leftmargin=*]

• We introduce two primal-dual projected-gradient controllers, for the case of single-integrator agents. The first is a continuous-time version of the algorithm in [23]. It employs a constant gain, whose choice requires the knowledge of the algebraic connectivity of the communication graph and of the Lipschitz and strong monotonicity constants of the game mapping. To relax this condition, we present a novel distributed averaging integral controller, that extends the solution of [8] to generalized games. In particular, the adaptive weights in place of the fixed global gain allows for a fully-decentralized tuning, that does not need any non-local information. For both algorithms, we prove convergence of primal and dual variables, under strong monotonicity and Lipschitz continuity of the game mapping. We are not aware of any other continuous-time GNE seeking scheme, for generally coupled costs games, whose convergence is guaranteed under such mild assumptions. (3)

• We propose a controller, with dynamic gains, specifically designed for generalized aggregative games. The agents keep and exchange an estimate of the aggregate value only, thus reducing communication and computation cost. With respect to [11], we can also handle inequality constraints. Furthermore, our algorithm requires no knowledge of global parameters and virtually no tuning. (§4)

• We show how all of our controllers can be adapted to learning GNE in games with shared constraints played by multi-integrator agents. To the best of our knowledge, we are the first to address generalized games with higher-order dynamical agents. Besides, the use of adaptive weights still ensures convergence without any a priori information on the game. (§5)

Some preliminary results of this paper have been submitted in [5], where algorithms with adaptive gains and aggregative games are not considered.

Basic notation: () denotes the set of (nonnegative) real numbers. For a differentiable function , is its gradient. (

) denotes a matrix/vector with all elements equal to

(); to improve clarity, we may add the dimension of these matrices/vectors as subscript.

denotes the identity matrix of dimension

. and

denote the transpose and the largest singular value of a matrix

, respectively. If is symmetric,

denote its eigenvalues.

stands for a symmetric positive definite matrix. denotes the block diagonal matrix with the matrices on its diagonal. denotes the Kronecker product of the matrices and . For , let and denote the Euclidean inner product and norm, respectively. Given vectors , we may denote , and, for each , .

Operator-theoretic definitions: A mapping is monotone (-strongly monotone, with ) if, for all , . A mapping is -Lipschitz continuous, with , if, for all , . denotes the closure of a set . Given a closed convex set , the mapping denotes the projection onto , i.e., . The set-valued mapping denotes the normal cone operator for the the set , i.e., if , otherwise. The tangent cone operator of is defined as , . denotes the projection on the tangent cone of at . By Moreau’s Decomposition Theorem [1, Th. 6.30], it holds that and , for any .

###### Lemma 1.

For any nonempty closed convex set , any and any , it holds that

 (y−y′)⊤ΠS(y,ξ)≤(y−y′)⊤ξ.

Thus, if , then

Proof. By Moreau’s theorem, , hence for any ,

## 2 Mathematical Background

We consider a group of noncooperative agents , where each agent shall choose its decision variable (i.e., strategy) from its local decision set . Let denote the stacked vector of all the agents’ decisions, the overall action space and . Moreover, let denote the collective strategy of all the agents, except that of agent . The goal of each agent is to minimize its objective function , which depends on both the local strategy and on the decision variables of the other agents . Furthermore, we address generalized games, where the coupling among the agents arises also via their feasible decision sets. In particular, we consider affine coupling constraints; thus the overall feasible set is

 X:=Ω∩{x∈Rn∣Ax≤b}, (1)

where and , with and being local data. The game then is represented by inter-dependent optimization problems:

 ∀i∈I:⎧⎨⎩argminyi∈RniJi(yi,x−i)s.t. (yi,x−i)∈X. (2)

In this paper, we consider the problem to compute a GNE, as formalized next.

###### Definition 1.

A collective strategy is a generalized Nash equilibrium if, for all ,

 x∗i∈argminyiJi(yi,x∗−i)s.t.(yi,x∗−i)∈X. □

Next, we formulate standard convexity and regularity assumptions for the constraint sets and cost functions.

###### Standing Assumption 1.

For each , the set is non-empty, closed and convex; is non-empty and satisfies Slater’s constraint qualification; is continuously differentiable and the function is convex for every .

Moreover, among all the possible GNE, we focus on the important subclass of v-GNE [12, Def. 3.11]. Under the previous assumption, is a v-GNE of the game in (2) if and only if there exist a dual variable such that the following KKT conditions are satisfied [12, Th. 4.8]:

 0n∈F(x∗)+A⊤λ∗+NΩ(x∗)0m∈−(Ax∗−b)+NmR≥0(λ∗), (3)

where is the pseudo-gradient mapping of the game:

 F(x):=col(∇xiJi(xi,x−i))i∈I. (4)

A sufficient condition for the existence of a unique v-GNE is the strong monotonicity of the pseudo-gradient [13, Th. 2.3.3], as postulated next. This assumption was used, e.g., in [17, Ass. 2], [4, Ass. 3], [8, Ass. 4].

###### Standing Assumption 2.

The pseudo-gradient mapping in (4) is -strongly monotone and -Lipschitz continuous, for some , .

## 3 Distributed generalized Nash equilibrium seeking

In this section,we consider the game in (2), where each agent is associated with the following dynamical system:

 ∀i∈I:˙xi=ΠΩi(xi,ui), xi(0)∈Ωi. (5)

Our aim is to design the inputs to seek a v-GNE in a fully distributed way. Specifically, each agent only knows its own cost function and feasible set . Besides, agent does not have full knowledge of , and only relies on the information exchanged locally with neighbors over a communication network . The unordered pair belongs to the set of edges if and only if agent and can exchange information. We denote the symmetric adjacency matrix of , with if , otherwise; the symmetric Laplacian matrix of ; the set of neighbors of agent

###### Standing Assumption 3.

The communication graph is undirected and connected.

In the remainder of the section, we present two dynamic controllers to asymptotically drive the system in (5) towards a v-GNE, in a fully-distributed fashion.

### 3.1 Distributed generalized Nash equilibrium seeking algorithm with constant gain

Our first algorithm is the continuous-time counterpart of [23, Alg. ]. To cope with partial-decision information, each agent keeps an estimate of all other agents’ action. We denote , where and is agent ’s estimate of agent ’s action, for all . Moreover, each agent keeps an estimate of the dual variable, and an auxiliary variable to allow for distributed consensus of the multipliers estimates. Our proposed dynamics are summarized in Algorithm 1, where is a global constant parameter and the initial conditions can be chosen arbitrarily.

We note that the agents exchange with their neighbors only, therefore the controller can be implemented distributedly. In steady state, agents should agree on their estimates, i.e., , for all . This motivates the presence of consensual terms for both primal and dual variables. We denote the consensual space of dimension , for an integer , and its orthogonal complement. Specifically, is the estimate consensus subspace and is the multiplier consensus subspace.

To write the closed-loop system in compact form, let us define, as in [23, Eq. 13-14], for all ,

 Ri: =[0ni×ni], (6a) Si: (6b)

where , . We note that selects the -th dimensional component from an -dimensional vector, while removes it. Thus, and . We define , . It follows that and . Moreover, we have that

 R⊤R+S⊤S=INn. (7)

Let , , , and, for any integer , . Furthermore, we define the extended pseudo-gradient mapping as:

 F(x):=col(∇xiJi(xi,xi−i)i∈I). (8)

Then, the closed-loop system, in compact form, reads as

 ˙x +S⊤(−cSLnx) (9a) ˙z =Lmλ (9b) ˙λ =ΠRNm≥0(λ,(ΛRx−b−Lmλ−Lmz)). (9c)

The following lemma relates the equilibria of the system in (9) to the v-GNE of the game in (2). The proof is analogous to [23, Th. 1], hence it is omitted.

###### Lemma 2.

The following statements hold:

1. [leftmargin=*]

2. Any equilibrium point of the dynamics in (9) is such that , , where the pair satisfies the KKT conditions in (3), hence is the v-GNE of the game in (2).

3. The set of equilibrium points of (9) is nonempty.

We remark that, in Algorithm 1, each agent evaluates the gradient of its cost function in its local estimate, not on the actual collective strategy. In fact, only when the estimates belong to the consensus space, i.e., (in the case of full information, for example), we have that . It follows that the operator is not necessarily monotone, not even if the pseudo gradient in (4) is strongly monotone (Standing Assumption 2). This is the main technical difficulty that arises when studying NE seeking under partial-information. To deal with this complication, cocoercivity of the extended pseudo-gradient (on the augmented estimate space) is sometimes postulated [30, Ass. 4], [27, Ass. 5]. However, this is a limiting assumption, which does not hold in general [23, Rem. 6]. Instead, our analysis is based on a weaker restricted monotonicity property, which can be guaranteed for any game satisfying Standing Assumptions 1-3, without additional hypotheses, as formalized in the next two statements.

###### Lemma 3.

The extended pseudo-gradient mapping in (8) is -Lipschitz continuous, for some : for any , .

Proof. See Appendix A

###### Lemma 4 ([23, Lem. 3]).

Let

 M1:=⎡⎢ ⎢⎣μN −θ0+θ2√N−θ0+θ2√N cλ2(L)−θ⎤⎥ ⎥⎦,c–:=(θ0+θ)2+4μθ4μλ2(L). (10)

For any , for any and any , it holds that and also that

 □
###### Remark 1.

Lipschitz continuity of the extended-pseudo gradient is often postulated, e.g. [23, Ass. 4], [8, Ass. 5]. Lemma 3 shows that this condition can be inferred by Standing Assumptions 1-2, similarly to [29, Lem. 1].

The restricted strong monotonicity property of the previous statement is not new in the context of of games played under partial-information, see, e.g., [23], [17], [26]. By leveraging Lemma 4, we next show the convergence of the dynamics in (9) to a v-GNE. For brevity of notation, let us define, in the remainder of the paper, the set

 Ω:={x∈RNn∣Rx∈Ω}. (11)
###### Theorem 1.

Let , with as in (10). For any initial condition in , the system in (9) has a unique Carathéodory solution, which belongs to for all . The solution converges to an equilibrium , with , , where the pair satisfies the KKT conditions in (3), hence is the v-GNE of the game in (2).

Proof. See Appendix E

### 3.2 Distributed generalized Nash equilibrium seeking algorithm with adaptive gains

The dynamic controller proposed in the last subsection allows to seek a v-GNE in a fully distributed way, provided that the global fixed gain is chosen high-enough, as in Theorem 1. However, selecting a gain that ensures convergence requires global knowledge about the graph , i.e., the algebraic connectivity, and about the game mapping, i.e., the strong monotonicity and Lipschitz constants. These parameters are unlikely to be available locally in a network system, when the cost function of each agent is private. To overcome this limitation and enhance the scalability of the design, the authors of [8] proposed a controller for the integrator systems in (5), where the gain is tuned online, thus relaxing the need for global information, for games without coupling constraints. In this section we extend their result to the GNE problem, i.e., to games with shared constraints.

Our proposed controller is given in Algorithm 2. For all , is the adaptive gain of agent , is a constant local parameter, , and the initial conditions can be chosen arbitrarily.

We can rewrite the overall closed-loop, in compact form, as

 ˙x =R⊤ΠΩ(Rx,−(F(x)+Λ⊤λ+R(LKL⊗In)x)) +S⊤(−S(LKL⊗In)x) (12a) ˙k =D(ρ)⊤(Γ⊗In)ρ,ρ=−(L⊗In)x (12b) ˙z =Lmλ (12c) ˙λ =ΠRNm≥0(λ,(ΛRx−b−Lmλ−Lmz)). (12d)

where , , , , .

###### Lemma 5.

The following statements hold:

1. [leftmargin=*]

2. Any equilibrium point of (12) is such that , , where the pair () satisfies the KKT conditions in (3), hence is the v-GNE of the game in (2).

3. The set of equilibrium points of (12) is nonempty.

Proof. See Appendix B

The following result is analogous to Lemma 4. The proof relies on the decomposition of along the consensus subspace , where is strongly monotone, and the disagreement subspace , where is strongly monotone.

###### Lemma 6.

Let

 (13)

For any and , for any and any , it holds that and also that

 □

Proof. See Appendix C Building on this property, we can now present the main result of this section.

###### Theorem 2.

For any initial condition in , the system in (12) has a unique Carathèodory solution, which belongs to for all . The solution converges to an equilibrium , with , and the pair satisfies the KKT conditions in (3), hence is the v-GNE of the game in (2).

Proof. See Appendix D

###### Remark 2.

Algorithm 2 allows for a fully uncoupled tuning. Specifically, each agent can choose locally the initial conditions and the rate , independently of the other agents and without any need for communication or knowledge of global parameters. Compared to Algorithm 1, the agents exchange some extra information, namely the variables .

## 4 Distributed generalized Nash equilibrium seeking for aggregative games

In this section, we focus on aggregative games. We assume that for all (hence ). In (average) aggregative games, the cost function of each agent depends on the local decision and on the value of the average strategy, i.e., It follows that, for each , there is a function such that the original cost function in (2) can be written as

 Ji(xi,x−i)=:fi(xi,avg(x)). (14)

Since an aggregative game is only a particular instance of the game in (2), all the considerations on the existence and uniqueness of a v-GNE and equivalence with the KKT conditions in (3) are still valid.

Moreover, Algorithms 1-2 could still be used to drive a system of single integrators (5) towards a v-GNE. This would require each agent to keep (and exchange) an estimate of all other agents’ action, i.e., a vector of components. In practice, however, the cost of each agent is only a function of the aggregative value , whose dimension is independent of the number of agents. To reduce the communication and computation burden, in this section, we introduce two distributed controllers, that are scalable with the number of agents, specifically designed to seek a v-GNE in aggregative games.

Our proposed dynamics are obtained by adapting Algorithms 1, 2 to take into account the aggregative structure of the game, and are illustrated in Algorithms 3, 4, respectively. Since the agents rely on local information only, they don’t have access to the actual value of the average strategy. Therefore, we embed each agent with an auxiliary error variable , that is an estimate of the quantity . Each agent aims at asymptotically reconstructing the true aggregate value, based on the information received from its neighbors. We used the notation

 ∇xifi(xi,σi)=∇yfi(y,σi)|y=xi+1N∇yfi(xi,y)|y=σi.

We note that, in Algorithms 3, 4, the agents send and receive the quantities , instead of exchanging the variables , like in Algorithms 1, 2.

Let , . Furthermore, let us define the extended pseudo-gradient mapping as

 ~F(x,σ):=col((∇xifi(xi,σi))i∈I). (15)
###### Lemma 7.

The mapping in (15) is -Lipschitz continuous, for some : for any , Therefore, the mapping is -Lipschitz continuous, for some , for all .

Proof. It follows from Lemma 3, by observing that .

We note that, in Algorithms 3, 4, each agent evaluates the gradient of its cost function in its local estimate of the average strategy. Only if all the estimates coincide with the actual value, i.e., , we can conclude that , as in (4).

The dynamics in Algorithms 3, 4 can be rewritten in compact form, as

 ˙x =ΠΩ(x,−~F(x,σ)−Λ⊤λ−cL¯nσ) (16a) ˙ς =−cL¯n(σ),σ=x+ς (16b) ˙z =Lmλ (16c) ˙λ =ΠRNm≥0(λ,(Λx−b−Lmλ−Lmz)), (16d)

and

 ˙x =ΠΩ(x,−~F(x,σ)−Λ⊤λ−(LKL⊗I¯n)σ) (17a) ˙ς =−(LKL⊗I¯n)σ,σ=x+ς (17b) ˙k =D(ρ)⊤(Γ⊗I¯n)ρ,ρ=−(L⊗I¯n)σ (17c) ˙z =Lmλ (17d) ˙λ =ΠRNm≥0(λ,(Λx−b−Lmλ−Lmz)), (17e)

respectively.

The convergence analysis of the dynamics in (16), (17), to a v-GNE makes use of an invariance property of the systems, namely that along any trajectory, if the initial conditions are chosen opportunely. In fact, it is crucial to set , that ensures . Indeed, the dynamics in (16b) or (17b) can be regarded as a continuous time dynamic tracking [20] for the time-varying quantity . By leveraging this invariance property, we obtain a refinement of the restricted strong monotonicity properties in Lemmas 4, 6, as demonstrated next.

###### Lemma 8 ([16, Lemma 4]).

Let

 (18)

For any , for any such that and any such that , it holds that , and also that

 (x−x′)⊤(~F(x,σ)−~F(x′,σ′))+(σ−σ′)⊤(cL¯n(σ−σ′))≥λmin(M3)∥∥col(x−x′,σ−1N⊗avg(x))∥∥2. □
###### Lemma 9.

Let

 M4=⎡⎢⎣μ −~θσ2−~θσ2 k∗λ2(L)2⎤⎥⎦,k––=~θ2σ4μλ2(L)2 (19)

For any and , for any such that and any such that , it holds that , and also that

 (x−x′)⊤(~F(x,σ)−~F(x′,σ′))+(σ−σ′)⊤((LK∗L⊗I¯n)(σ−σ′))≥λmin(M4)∥∥col(x−x′,σ−1N⊗avg(x))∥∥2. □

Proof. See Appendix F

We are now ready to prove the main results of this section.

###### Theorem 3.

Let , with as in (18). For any initial condition in such that , the system in (16) has a unique Carathèodory solution that belongs to for all . The solution converges to an equilibrium , with , , where the pair satisfies the KKT conditions in (3), hence is the v-GNE of the game in (2).

Proof. See Appendix H.

###### Theorem 4.

For any initial condition in such that , the system in (17) has a unique Carathèodory solution, which belongs to for all . The solution converges to an equilibrium , with ,