I Introduction
Aggregative game theory
[1] is a mathematical framework to model the interdependent optimal decision making problems for a set of noncooperative agents, whenever the decision of each agent is affected by some aggregate effect of all the agents. This feature emerges in several application areas, such as demand side management in the smart grid [2], e.g. for electric vehicles [3, 4] and thermostatically controlled loads [5, 6], demand response in competitive markets [7] and network congestion control [8].Existence and uniqueness of Nash equilibria in (aggregative) noncooperative games is well established in the literature of operation research [9], [10, §12], and automatic control [11, 12]. For the computation of a game equilibrium, several algorithms are available, both distributed protocols [13, 14] and semidecentralized schemes [15, 16, 17, 18]. Among these, an elegant approach is to characterize the desired equilibrium solutions as the zeros of an operator, possibly monotone, e.g. the concatenation of interdependent Karush–Kuhn–Tucker operators, and in turn formulate an equivalent fixedpoint problem, which is solved via appropriate fixedpoint iterations. Overall, the available methods can ensure global convergence to an equilibrium if the coupling among the cost functions of the agents is “well behaved”, e.g. if the problem data are convex and the socalled pseudogradient game mapping is (strictly, strongly) monotone [19].
A popular class of algorithms for Nash equilibrium seeking is that of projectedgradient algorithms [20, §12], [14, 16, 19]. Whenever the pseudogradient game mapping is strongly monotone, projectedgradient algorithms can ensure fast convergence to a Nash equilibrium, possibly via distributed computation and information exchange. It follows that projectedgradient methods have the potential to be fast, simple and scalable with respect to the population size. At the same time, in the context of Nash equilibrium seeking, the convergence analyses for the available projectedgradient methods are quite diverse in nature.
In this paper, we aim at a unifying convergence analysis for projectedgradient algorithms that are adopted for the computation of generalized Nash equilibria in aggregative games. Specifically, we adopt a general perspective based on monotone operator theory [21] to show that projectedgradient algorithms with sequential updates belong to the class of preconditioned forwardbackward splitting methods, introduced in [22] for multiagent network games.
The main technical contribution of the paper is to conceive a design procedure for the preconditioned forwardbackward splitting method. The proposed design is based not only on the splitting of the monotone operator whose zeros are the game equilibria, but also on the choice of the socalled preconditioning matrix, which induces the quadratic norm adopted to show global convergence of the resulting algorithm. Since the convergence characterization of the forwardbackward splitting method is well established, the advantage of the proposed design is that global convergence follows provided that some mild monotonicity assumptions on the problem data are satisfied.
Remarkably, we discover that two recent projectedgradient algorithms for Nash equilibrium seeking in aggregative games, [16] and [14], can be equivalently written as preconditioned forwardbackward splitting methods with symmetric preconditioning matrix, despite their algorithmic formulation is “asymmetric”.
Basic notation
denotes the set of real numbers, and the set of extended real numbers. (
) denotes a matrix/vector with all elements equal to
(); to improve clarity, we may add the dimension of these matrices/vectors as subscript. denotes the Kronecker product between matrices and ;denotes the maximum singular value of
;denotes the set of eigenvalues of
. Given vectors , .Operator theoretic definitions
denotes the identity operator. The mapping denotes the indicator function for the set , i.e., if , otherwise. For a closed set , the mapping denotes the projection onto , i.e., . The setvalued mapping denotes the normal cone operator for the the set , i.e., if , otherwise. For a function , ; denotes its subdifferential setvalued mapping, defined as ; A setvalued mapping is Lipschitz continuous, with , if for all , , ; is (strictly) monotone if for all , , ; is strongly monotone, with , if for all , , ; is averaged, with , if , for all ; is cocoercive, with , if is averaged. With , we denote the resolvent operator of , which is averaged if and only if is monotone; and denote the set of fixed points and of zeros, respectively.
Ii Generalized aggregative games
Iia Mathematical formulation
We consider a set of noncooperative agents, where each agent shall choose its decision variable (i.e., strategy) from the local decision set with the aim of minimizing its local cost function , which depends on both the local variable (first argument) and on the decision variables of the other agents, (second argument).
We focus on the class of aggregative games, where the cost function of each agent depends on the local decision variable and on the value of the aggregation function , with . In particular, we consider average aggregative games, where the aggregation function is the average function, i.e.,
(1) 
Thus, for each , there is a function such that the local cost function can be written as
(2) 
Furthermore, we consider generalized games, where the coupling among the agents arises not only via the cost functions, but also via their feasible decision sets. In our setup, the coupling constraints are described by an affine function, , where and . Thus, the collective feasible set, , reads as
(3) 
while the feasible decision set of each agent is characterized by the setvalued mapping , defined as where and . The set represents the local decision set for agent , while the matrix defines how agent is involved in the coupling constraints. For instance, the shared constraints in (3) may contain a sparsity pattern that can be defined via a graph, where each agent has a set of “neighbors” with whom to share some constraints.
Remark 1 (Affine coupling constraint)
Affine coupling constraints as considered in this paper are very common in the literature of noncooperative games, see [16], [17], [22], [23]. Moreover, we recall that the more general case with separable convex coupling constraints can be reformulated as game via affine coupling constraints [24, Remark 2].
Next, let us postulate standard convexity and compactness assumptions for the constraint sets, convexity and differentiability assumptions for the local cost functions.
Standing Assumption 1 (Convex differentiable functions)
For each and the function is convex and continuously differentiable.
Standing Assumption 2 (Compact convex constraints)
For each , the set is nonempty, compact and convex. The set satisfies the Slater’s constraint qualification.
In summary, the aim of each agent , given the decision variables of the other agents, , is to choose a strategy, , that solves its local optimization problem according to the game setup previously described, i.e.,
(4) 
From the gametheoretic perspective, we consider the problem to compute a Nash equilibrium, as formalized next.
Definition 1 (Generalized Nash equilibrium)
The collective strategy is a generalized Nash equilibrium (GNE) of the game in (4) if and for all
In other words, a set of strategies is a Nash equilibrium if no agent can improve its objective function by unilaterally changing its strategy to another feasible one.
IiB Variational equilibria and pseudogradient of the game
Within all the possible Nash equilibria, we focus on an important subclass of equilibria, with some relevant structural properties, such as “larger social stability” and “economic fairness” [9, Theorem 4.8], that corresponds to the solution set of an appropriate variational inequality. Let us first formalize the notion of variational inequality problem.
Definition 2 (Generalized variational inequality)
Consider a closed convex set , a setvalued mapping , and a singlevalued mapping . The generalized variational inequality problem GVI, is the problem to find and such that
If for all , then GVI reduces to the variational inequality problem VI.
A fundamental mapping in a noncooperative game is the socalled pseudogradient, , defined as
(5) 
Namely, the mapping is obtained by stacking together the subdifferentials of the agents’ objective functions with respect to their local decision variables.
Under Assumptions 12, it follows by [10, Proposition 12.4] that any solution of GVI is a Nash equilibrium of the game in (4). The inverse implication is not true in general, and actually in passing from the Nash equilibrium problem to the GVI problem most solutions are lost [10, §12.2.2]; indeed, a game may have a Nash equilibrium while the corresponding GVI has no solution. Note that, since the cost functions are differentiable (by Assumption 2), then GVI reduces to VI, which is commonly addressed in the context of game theory via projected gradient algorithms [14, 16, 19], [20, §12].
Under the postulated standing assumptions, it is shown in [10, Proposition 12.11] that a sufficient condition for the existence (and uniqueness) of a variational GNE of the game in (4) is the (strict) monotonicity of the pseudogradient in (5). Thus, let us assume strongly monotonicity of .
Standing Assumption 3 (Strong monotonicity)
The pseudogradient in (5) is strongly monotone and Lipschitz continuous, for some constants .
Iii Generalized Nash equilibrium as zero of the sum of two monotone operators
In this section, we exploit operator theory to recast the Nash equilibrium seeking problem into a monotone inclusion, namely, the problem of finding a zero of a setvalued monotone operator. As first step, we characterize a GNE of the game in terms of KKT conditions of the coupled optimization problems in (4). For each agent , let us introduce the Lagrangian function , defined as
where is the Lagrangian multiplier associated with the coupling constraints. It follows from [10, §12.2.3] that the set of strategies is a GNE of the game in (4) if and only if the following coupled KKT conditions are satisfied:
(6) 
The constraint qualification in Assumption 2 is needed at this stage to ensures boundedness of the dual variables ’s.
In a similar fashion, we characterize a variational GNE in term of KKT conditions by exploiting the Lagrangian duality scheme for the corresponding VI problem, see [25, §3.2]. Specifically, is a solution of VI if and only if Then, the associated KKT optimality conditions read as
(7) 
To cast (7) in compact form, we introduce the setvalued mapping , defined as
(8) 
Essentially, the role of the mapping is that its zeros correspond to the variational generalized Nash equilibria of the game in (4), as formalized in the next statement.
Proposition 1 ([18, Th. 1])
To conclude this section, we note that the mapping can be written as the sum of two operators, defined as
(9)  
(10) 
The formulation is called splitting of , and will be exploited in different ways later on. We show next that the mappings and are both monotone, which paves the way for splitting algorithms.
First, consider in (10). The first term is maximally monotone, since normal cones of closed convex sets are maximally monotone and the concatenation preserves maximality [21, Prop. 20.23]; the second term
is linear and skew symmetric, i.e.,
, thus maximally monotone [21, Ex. 20.30]. Then, the maximal monotonicity of follows from [21, Cor. 24.4], since . To prove that is cocoercive, we note that for all , it holds thatThe first and second inequalities follow from the strong monotonicity and Lipschitz continuity, respectively, of the mapping , postulated in Assumption 3.
Iv Preconditioned ForwardBackward splitting
In light of Lemma 1, the forwardbackward (FB) splitting [21, §25.3] guarantees convergence to a zero of . In this section, we discuss a design procedure for FB algorithms, which is particularly useful when the resolvent of the operator cannot be computed explicitly. Moreover, we show that two existing algorithms for GNE seeking in aggregative games belong to this class of algorithms.
Iva Preconditioned ForwardBackward: Design Procedure
The main idea of the FB splitting is that the zeros of the mapping in (8) correspond to the fixed points of a certain operator which depends on the chosen splitting (10)(9), as formalized next.
Lemma 2
For any matrix , the following equivalence holds:
(11) 
where and .
Consider a vector , then
where the first equivalence holds since .
The FB algorithm is the Banach–Picard iteration [21, (1.67)] applied to the mappings in (11), i.e.,
(12) 
In numerical analysis, represents a forward step with size and direction defined by , while represents a backward step. Directly from the iteration in (12), we have that
(13) 
The choice of the preconditioning matrix in (13) plays a key role in the algorithm design. Next, we provide some general guidelines to design .
Design guidelines for the preconditioning matrix :

, (necessary);

in (13) explicitly computable (necessary);

(convenient convergence analysis);

iterations in (12) sequential (convenient implementation).
Without loss of generality, we denote , then the inclusion in (13) reads in expanded form as
(14) 
Consider a symmetric matrix , designed accordingly to the guidelines above, i.e.,
(15) 
where and the coefficients (step sizes) and are chosen such that has positive eigenvalues (guideline 1), as formalized in the next lemma.
Lemma 3
The conditions in (16) directly follow by applying the the Schur’s complement on in (15). Now, we present the preconditioned ForwardBackward (pFB) algorithm associated with , which is the Banach–Picard iteration in (12) for .
Algorithm 1: Preconditioned Forward Backward (pFB)Remark 2
The iterations of Algorithm 1 are sequential (guideline 4), namely, the multiplier update, , exploits the most recent value of the agents’ strategies, .
In the next statement, we show the convergence of Algorithm 1 to a variational generalized Nash equilibrium, under suitable choices of the step sizes.
Theorem 1 (Global convergence of pFB)
The sequence defined by Algorithm 1, with step sizes , for all , and , with , globally converges to some , with as in (8).
See Section V.
IvB The Asymmetric Projection Algorithm in [16, Alg. 1] is a preconditioned ForwardBackward splitting
We note that Algorithm 1 (with equal step sizes) corresponds to the “asymmetric” projected algorithm (APA) proposed in [16, Alg. 1]. Therein, the algorithm design and its convergence analysis rely on a variational inequality formulation of the Nash equilibrium problem. Specifically, the authors define the convex set and the monotone mapping
and characterize the GNE as solutions of VI. Then, to solve VI in a semidecentralized fashion, the authors propose an asymmetric implementation of the projection algorithm for variational inequalities [20, 12.5.1], in which each iteration is computed as
(17) 
where and
(18) 
If the parameter in is chosen such that , then the unique solution in (17) is , where is the projection operator characterized by the asymmetric matrix in (18). Thus, the iteration in (17) equivalently reads as
(19) 
which is nothing but a pFB associated with the splitting and the preconditioning matrix .
Remark 3
With (19), we showed that pFB algorithms based on different splittings and preconditioning matrices can lead to the same algorithm. From an operatortheoretic perspective, the convergence analysis is more convenient for the pFB with symmetric matrix (guideline 3), since the properties that , have with the standard inner product are preserved for , with inner product , as shown in Section V.
IvC The distributed algorithm for aggregative games on graphs in [14, §3] is a preconditioned ForwardBackward
In this section, we show that the synchronous distributed algorithm for NE seeking in network aggregative games proposed in [14, §3] can be written as a pFB splitting.
In [14], the authors consider an aggregative game without coupling constraints, i.e., for all , and wherein the agents have no access to the aggregate decision (1
), but build an estimate of it by communicating over an undirected network with their neighboring agents.
Specifically, let be the set of underlying undirected edges between agents; let denote the set of neighbors of agent , with the convention that ; let be the degree matrix, where ; let be the adjacency matrix, such that if , otherwise. let be the Laplacian matrix and let us define the matrix , with , such that, given a vector , with , then
Let be the extension of in (5) to the augmented space of actions and estimates, defined as
(20) 
Note that . Next, we present a static version of the algorithm in [14, §3], whose convergence to a NE is established in [14, Prop. 2].
Algorithm 2: Koshal–Nedić–Shanbhag algorithm
In the following statement, we show that Algorithm 2 is a pFB splitting with symmetric preconditioning matrix.
Proposition 2
V Convergence Analysis
First, we show that the properties the mappings , have with the standard inner product are preserved for , with the inner product .
Lemma 4
The mappings , , , satisfy the following properties in the induced norm:

is cocoercive and is averaged, where
. 
is maximally monotone and is averaged.
(i): We need to show that for all the following condition holds:
(24) 
We first provide an upper bound for the right hand side of (24). Let us denote for , then
(25) 
where is symmetric and positive definite if the step sizes , are chosen as in Lemma 3. Moreover, it holds that , where is the smallest eigenvalue of . Now, we exploit the cocoercivity of and the upper bound in (25) to define the cocoercivity constant in (24).
Thus, the mapping is cocoercive with constant w.r.t. the induced norm. Since is cocoercive, it follows from [21, Prop. 4.33] that is averaged. (ii): Since is maximally monotone by Lemma (1) and is positive definite, if the step sizes are chosen as in Lemma 3, then the maximal monotonicity of follows from [26, Lemma 3.7].
Next, we show that the FB operator is averaged if the step sizes are chosen small enough.
Lemma 5
By Lemma 4, and are averaged with constants and , respectively. If , then is averaged with , by [27, proposition 2.4]. To conclude, we note that the following condition implies that :
(28) 
where the second inequality in (28) holds for step sizes chosen as in (26). Moreover, if the step sizes are equal, i.e., for all , then holds for as in (27). We can now prove the convergence of Algorithm 1.
Proof of Theorem 1: The iterations in Alg. 1 are obtained explicitly by substituting into (14) and solving for , . Thus, Alg. 1 is the Banach–Picard iteration of the mapping , which is averaged, with , by Lemma 5, if the step sizes satisfy (26). The convergence of the sequence generated by the Banach–Picard iteration of to follows by [21, Prop. 15.5].
Vi Conclusion
By monotone operator theory, projectedgradient methods for generalized Nash equilibrium seeking in aggregative games are preconditioned forwardbackward splitting methods, whose convergence has been established for problems with strongly monotone pseudogradient mapping.
References
 [1] N. S. Kukushkin, “Best response dynamics in finite games with additive aggregation,” Games and Economic Behavior, vol. 48, no. 1, pp. 94–10, 2004.
 [2] W. Saad, Z. Han, H. Poor, and T. Başar, “Game theoretic methods for the smart grid,” IEEE Signal Processing Magazine, pp. 86–105, 2012.
 [3] F. Parise, M. Colombino, S. Grammatico, and J. Lygeros, “Mean field constrained charging policy for large populations of plugin electric vehicles,” in Proc. of the IEEE Conference on Decision and Control, Los Angeles, California, USA, 2014, pp. 5101–5106.
 [4] Z. Ma, S. Zou, L. Ran, X. Shi, and I. Hiskens, “Efficient decentralized coordination of largescale plugin electric vehicle charging,” Automatica, vol. 69, pp. 35–47, 2016.
 [5] S. Grammatico, B. Gentile, F. Parise, and J. Lygeros, “A mean field control approach for demand side management of large populations of thermostatically controlled loads,” in Proc. of the IEEE European Control Conference, Linz, Austria, 2015.
 [6] S. Li, W. Zhang, J. Lian, and K. Kalsi, “Marketbased coordination of thermostatically controlled loads  Part I: A mechanism design formulation,” IEEE Trans. on Power Systems, vol. 31, no. 2, pp. 1170–1178, 2016.
 [7] N. Li, L. Chen, and M. A. Dahleh, “Demand response using linear supply function bidding,” IEEE Transactions on Smart Grid, vol. 6, no. 4, pp. 1827–1838, 2015.
 [8] J. Barrera and A. Garcia, “Dynamic incentives for congestion control,” IEEE Trans. on Automatic Control, vol. 60, no. 2, pp. 299–310, 2015.
 [9] F. Facchinei and C. Kanzow, “Generalized nash equilibrium problems,” Annals of Operations Research, vol. 175, no. 1, pp. 177–211, 2010.
 [10] D. P. Palomar and Y. C. Eldar, Convex optimization in signal processing and communications. Cambridge university press, 2010.
 [11] L. Pavel, “An extension of duality to a gametheoretic framework,” Automatica, vol. 43, pp. 226 –237, 2007.
 [12] A. A. Kulkarni and U. Shanbhag, “On the variational equilibrium as a refinement of the generalized Nash equilibrium,” Automatica, vol. 48, pp. 45 –55, 2012.
 [13] F. Salehisadaghiani and L. Pavel, “Distributed Nash equilibrium seeking: A gossipbased algorithm,” Automatica, vol. 72, pp. 209–216, 2016.
 [14] J. Koshal, A. Nedić, and U. Shanbhag, “Distributed algorithms for aggregative games on graphs,” Operations Research, vol. 64, no. 3, pp. 680–704, 2016.
 [15] S. Grammatico, F. Parise, M. Colombino, and J. Lygeros, “Decentralized convergence to Nash equilibria in constrained deterministic mean field control,” IEEE Trans. on Automatic Control, vol. 61, no. 11, pp. 3315–3329, 2016.
 [16] D. Paccagnan, B. Gentile, F. Parise, M. Kamgarpour, and J. Lygeros, “Distributed computation of generalized Nash equilibria in quadratic aggregative games with affine coupling constraints,” in Proc. of the IEEE Conf. on Decision and Control, Las Vegas, USA, 2016.
 [17] S. Grammatico, “Dynamic control of agents playing aggregative games with coupling constraints,” IEEE Trans. on Automatic Control, vol. 62, no. 9, pp. 4537 – 4548, 2017.
 [18] G. Belgioioso and S. Grammatico, “Semidecentralized Nash equilibrium seeking in aggregative games with coupling constraints and nondifferentiable cost functions,” IEEE Control Systems Letters, vol. 1, no. 2, pp. 400–405, 2017.
 [19] ——, “On convexity and monotonicity in generalized aggregative games,” IFACPapersOnLine, vol. 50, no. 1, pp. 14 338–14 343, 2017.
 [20] F. Facchinei and J. Pang, Finitedimensional variational inequalities and complementarity problems. Springer Verlag, 2003.
 [21] H. H. Bauschke and P. L. Combettes, Convex analysis and monotone operator theory in Hilbert spaces. Springer, 2011.
 [22] P. Yi and L. Pavel, “A distributed primaldual algorithm for computation of generalized nash equilibria via operator splitting methods,” in 56th IEEE Annual Conference on Decision and Control, CDC 2017, Melbourne, Australia, December 1215, 2017, 2017, pp. 3841–3846.
 [23] S. Liang, P. Yi, and Y. Hong, “Distributed nash equilibrium seeking for aggregative games with coupled constraints,” Automatica, vol. 85, pp. 179–185, 2017.
 [24] S. Grammatico, “Proximal dynamics in multiagent network games,” IEEE Transactions on Control of Network Systems, 2017.
 [25] A. Auslender and M. Teboulle, “Lagrangian duality and related multiplier methods for variational inequality problems,” SIAM Journal on Optimization, vol. 10, no. 4, pp. 1097–1115, 2000.
 [26] P. L. Combettes and B. C. Vũ, “Variable metric forward–backward splitting with applications to monotone inclusions in duality,” Optimization, vol. 63, no. 9, pp. 1289–1318, 2014.
 [27] P. L. Combettes and I. Yamada, “Compositions and convex combinations of averaged nonexpansive operators,” Journal of Mathematical Analysis and Applications, pp. 55–70, 2015.
Comments
There are no comments yet.