Continuous-time integral dynamics for Aggregative Game equilibrium seeking

03/28/2018 ∙ by Claudio De Persis, et al. ∙ University of Groningen Delft University of Technology 0

In this paper, we consider continuous-time semi-decentralized dynamics for the equilibrium computation in a class of aggregative games. Specifically, we propose a scheme where decentralized projected-gradient dynamics are driven by an integral control law. To prove global exponential convergence of the proposed dynamics to an aggregative equilibrium, we adopt a quadratic Lyapunov function argument. We derive a sufficient condition for global convergence that we position within the recent literature on aggregative games, and in particular we show that it improves on established results.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Aggregative game theory [1] represents a mathematical framework to model inter-dependent optimization problems for a set of noncooperative agents, or players, whenever the decision of each agent is affected by some aggregate effect of all the agents. Remarkably, this feature emerges in several application areas, such as demand side management in the smart grid [2], e.g. for electric vehicles [3, 4, 5] and thermostatically controlled loads [6, 7], demand response in competitive markets [8] and network congestion control [9, 10].

Existence and uniqueness of equilibria in (aggregative) games has been widely studied in the literature [11, 12], [13, §12], [14, Part II], also in automatic control and control systems[15, 16, 17]. To compute a game equilibrium, severable algorithms are available, both distributed [18, 19, 20, 21, 22] and semi-decentralized [23, 24, 10, 25, 26].

The majority of the available algorithms consider discrete-time dynamics that, under appropriate technical assumptions and sufficient conditions on the problem data, converge to an equilibrium of the game, e.g. Nash equilibrium. An elegant approach is in fact to characterize the desired equilibrium solutions as the zeros of a monotone operator, e.g. the concatenation of interdependent Karush–Kuhn–Tucker operators, and in turn formulate an equivalent fixed-point problem, which is solved via appropriate discrete-time nonlinear dynamics with guaranteed global asymptotic convergence.

On the other hand, only a few contributions have addressed the equilibrium computation problem via continuous-time dynamics, e.g. [21, 22]. One reason is that in the continuous-time case where optimal decision problems of the agents are subject to constraints, projected dynamics, hence dynamics with discontinuous right-hand side, shall be considered. Specifically, in [21], the authors propose distributed dynamics based on a consensus protocol to compute a Nash equilibrium, under technical assumptions that include the strong monotonicity of the so-called pseudo-gradient game mapping, but in the absence of constraints. In [22]

, the authors address the generalized Nash equilibrium problem, that is, with both local and coupling constraints, via extended continuous-time dynamics that include three auxiliary vector variables for each agent. To prove convergence to an equilibrium, the authors postulate that the pseudo-gradient mapping is strictly monotone and that the parameter gains are chosen large enough, that is, directly proportional to the number of agents in the game.

Differently from the available literature, and in particular from [21, 22], the contribution of this paper is to provide a first, simple, integral control algorithm for the computation of an aggregative equilibrium via semi-decentralized dynamics. Since we consider games with constraints, we propose equilibrium seeking dynamics that are described as a projected dynamical system [27]. Therefore, the technical difficulty we address is to study the solutions to the derived projected dynamical system in the sense of Carathèodory solutions. With this aim, we exploit an invariance principle for projected dynamical systems. The main technical contribution is then to prove global exponential convergence of the proposed continuous-time, semi-decentralized, dynamics to an equilibrium of the considered aggregative game, under a mild design choice that involves a few problem parameters only. Interestingly, we discover that the derived sufficient condition improves on that in [24, Th. 2], especially when the number of agents playing the aggregative game is large.

The paper is organized as follows. Section 2 introduces the mathematical setup and defines the aggregative equilibrium as solution concept. In Section 3, we propose integral dynamics for the equilibrium computation and present the main technical results. In Section 4, we compare the derived sufficient condition for global convergence within the recent literature of (monotone) aggregative games. Section 6 summarizes the message of the paper and points at future research directions. The proofs are given in the Appendix to improve the reading flow.

Basic notation

denotes the set of real numbers, while the set of extended real numbers.

denotes the identity matrix.

() denotes a matrix/vector with all elements equal to (); to improve clarity, we may add the dimension of these matrices/vectors as subscript. denotes the Kronecker product between matrices and .

denotes the maximum singular value of matrix

. Given vectors , we define , and . Given sets , we define the Cartesian product by .

Definitions

Let the set be non-empty. The mapping denotes the indicator function, i.e., if , otherwise. The set-valued mapping denotes the normal cone operator, i.e., if , otherwise. The mapping denotes the projection operator; denotes the projection of the vector onto the tangent cone of at .

For a function , ; denotes its subdifferential set-valued mapping, defined as ; if is differentiable at , then .

Given a closed convex set and a single-valued mapping , the variational inequality problem VI, is the problem to find such that

2 Technical background: Aggregative games and variational aggregative equilibria

An aggregative game, , consists of agents (or players) indexed by the set , where each agent can decide on a vector , with the aim to minimize its cost function , , where .

Let us postulate the following standing assumption that is valid throughout the paper.

Standing Assumption 1

Compactness and convexity, from [24, Assumptions 1, 2]. For each , the set is nonempty, compact and convex; the cost function is defined as

(1)

where the function is twice continuously differentiable and -strongly convex in , with constant , and .

Remark 1

In a discrete-time setting, the functions can be assumed to be continuous, not necessarily differentiable [24], [10]. Since the focus of this paper is equilibrium seeking in continuous-time, to avoid technical difficulties with non-smooth analysis and set-valued subdifferential mappings, we assume that the functions are twice continuously differentiable, as in [21, 22]. We envision, however, that the differentiability assumption can be relaxed via the arguments in [28].

In this paper, we focus on the design of continuous-time dynamics that converge to an aggregative equilibrium, which is a set of decision variables such that each is optimal given the average among all the decision variables.

Definition 1

Aggregative equilibrium, from [10, Def. 1]. A collective vector is an aggregative equilibrium for the game if, for all ,

The set of aggregative equilibria is denoted by .

Remark 2

Alternatively, instead of the aggregative equilibrium, the notion of Nash equilibrium can be considered as target collective set, see [24, Sec. V]. Moreover, we refer the interested reader to [10, Th. 1] and to [29, 30] for a comparison between aggregative and Nash equilibria.

It follows from [10, Prop. 1] that an aggregative equilibrium does exist. In view of Standing Assumption 1, we provide an equivalent characterization of aggregative equilibrium via a variational inequality.

Lemma 1

Variational characterization, from [31, Prop. 1.4.2]. A collective vector is an aggregative equilibrium for the aggregative game if and only if it satisfies the variational inequality

(2)

In view of the equilibrium characterization in Lemma 1, uniqueness of the aggregative equilibrium can be studied via variational inequality arguments, see [32, Th. 3].

3 Continuous-time integral dynamics for aggregative equilibrium seeking

With the aim to asymptotically reach an aggregative equilibrium, we propose the continuous-time dynamics

(3)

which reads in projected vector form as

(4)

where and is a free design parameter.

We note that the structure of the computation and information exchange in (3) is semi-decentralized: the agents perform decentralized computations, namely, projected-gradient steps, and do not exchange information among each other, while a central control unit, which does not participate in the game, collects the average of the agents’ decisions (aggregate information), , and broadcasts a signal, , to the agents playing the aggregative game. In turn, the dynamics of the broadcast signal are driven by the aggregate information, .

First, we observe that an equilibrium for the dynamics in (3) generates an aggregative equilibrium.

Lemma 2

If the pair is an equilibrium for the dynamics in (3), then is an aggregative equilibrium for the game . Conversely, if is an aggregative equilibrium for the game , then the pair is an equilibrium for the dynamics in (3).

In the rest of the section, we analyze the convergence of the projected dynamics in (4) to an equilibrium. In particular, we take two preliminary steps. First, we introduce a quadratic storage function, , and analyze its Lyapunov derivative, . Then, we consider a quadratic Lyapunov function candidate, , and establish a condition on the problem data , , and on the design parameter such that its Lyapunov derivative is negative definite. Finally, we establish our main asymptotic convergence result, whose proof - given in the appendix - is based on Lyapunov stability theory for projected dynamical systems with Carathèodory solutions.

Lemma 3

Storage Lyapunov function. The function

is such that, for all and ,

(5)

where .

Proposition 1

Lyapunov decrease. Consider the Lyapunov function candidate defined as

If the inequality condition

(6)

holds, then we have that

(7)

for all , for some .

Theorem 1

Global exponential convergence. Assume that the condition in (6) holds. For any initial condition , there exists a unique Carathèodory solution to (4) starting from , which remains in for all time, and exponentially converges to the set .

4 Discussion on the convergence condition

For , the -dynamics in (4) can be thought as a control law, implemented by a central agent, which is the continuous-time counterpart of the Banach–Picard iteration studied in [24], while for , the counterpart of the Krasnoselskij iteration.

We note however that in the continuous-time dynamics in (4), the parameter gain must not necessarily be smaller than . For instance, if , for some , then the inequality condition in (6) reads as , hence , which is equivalent to whenever . Let us compare the latter with the inequality condition established in [24, Th. 2], i.e., . Since , the inequality condition in (6) is less strict than that in [24, Th. 2] if , i.e., , e.g. for large number of agents .

Next, we relate the condition in (6) with the monotonicity of the mapping that defines the variational inequality in (2), Lemma 1, which is

(8)

In view of [33, Cor. 1] and [23, Lemma 3], since the local cost functions are -strongly convex, we have that the mapping is strictly monotone if , e.g. if .

Finally, differently from the convergence condition provided in [22, Sec. 4], where the parameter gains must be chosen directly proportional to the number of agents, , the inequality condition in (6) becomes less strict as grows. Clearly, the latter is an advantageous feature especially when the number of agents is very large.

5 Numerical simulation: Illustrative game-theoretic demand side management

A recurrent application area for aggregative games is the demand side management, where the agents of the game are electricity prosumers, that is, consumers and possibly producers, whose disutility function depends on the average electricity demand, e.g. via a common electricity price.

For instance, in [22, Sec. 5.2], the individual cost function of each agent is chosen as

for some reference decision variable , parameter , parameters and . Since the dependence of on its second argument is affine, the setup satisfies our Standing Assumption 1, precisely with .

Numerically, we take the following values: , , , , sampled uniformly in and for all . For three choices of , i.e., , , or , we plot the time evolution of the distance of the average strategy from its equilibrium, (Fig. 1) and that of the integral control signal from its equilibrium (Fig. 2).

Figure 1: Distance between the average strategy over time and the average strategy at the aggregative equilibrium.
Figure 2: Distance between the integral control signal over time and the average strategy at the aggregative equilibrium.

In our numerical experience, the control gain in (4) can sensibly speed up the convergence to an equilibrium, but only up to a critical value above which oscillations may occur. Indeed, the inequality in (6) does not hold for large .

6 Conclusion and outlook

In the context of aggregative games, continuous-time integral dynamics with semi-decentralized computation and information exchange can ensure global asymptotic convergence to aggregative equilibria, provided that the control gain is appropriately chosen.

Future research will focus on extending the convergence analysis to the case of general network games, possibly with coupling constraints, and with relaxed technical assumptions.

7 Acknowledgments

The authors would like to thank Mr. G. Belgioioso (TU Eindhoven), Mr. T.W. Stegink and Mr. T. Van Damme (U. Groningen) for valuable discussions on related topics.

Appendix: Proofs

Proof of Lemma 1

By convexity and the minimum principle, a collective vector is an aggregative equilibrium exists if and only if

In view of (1), the variational inequality above can be equivalently rewritten as in (2).

Proof of Lemma 2

Let be an equilibrium for (3), i.e.

or, equivalently,

If , then , for all , hence the equilibrium is such that . This equation implies that satisfies the variational inequality in (2), hence, by Lemma 1, it is an aggregative equilibrium.

If , then there exists a nonempty subset of indices such that for all . For each , we necessarily have (see [34]) that

and by definition of the normal cone operator , it follows that, for all ,

On the other hand, for each , we have that , hence . We conclude that even when , the variational inequality in (2) holds and, by Lemma 1, is an aggregative equilibrium.

Conversely, if is an aggregative equilibrium, then for all . Since for all , we have that . The thesis then follows by setting .

Proof of Lemma 3

We adapt the approach in [34] to the considered storage function . Since , we have that

(9)

where denotes the th block-component of the vector . Now, as a consequence of Moreau’s decomposition theorem, it holds that

(10)

and, by definition of and noting that , we have that

(11)

From the latter and (9), it then follows that

(12)

Now, we note that since is the set of aggregative equilibria, is an aggregative equilibrium, and by Lemma 2, the pair is an equilibrium for the dynamics in (3), hence it satisfies

(13)

where . Again by Moreau’s decomposition theorem, the latter yields

(14)

which, by definition of , implies that

(15)

We conclude that

(16)

where the first equality holds by (12), and the second one holds in view of (15). The thesis then follows by summing up the inequalities over .

Proof of Proposition 1

For ease of notation, let us define . We first note that

(17)

where we have used Lemma 3 with .

Since the functions are twice continuous differentiable and the sets ’s are convex, we have that

(18)

for some , . By -strong convexity of on , i.e., for all , we derive that

(19)

Furthermore, we have that

(20)

and

(21)

By rearranging the terms in matrix form, we obtain that

with matrix

Finally, we apply the Gershgorin theorem to the matrix . Namely, the condition in (6) implies that

which in turn implies that . The proof then follows with .

Proof of Theorem 1

The dynamics in (4) represent a projected dynamical system with discontinuous right-hand side [35]. Thus, its solutions must be intended in a Carathèodory sense.

By [35, Th. 2.5] and [36, Prop. 2.2], since the mapping is Lipschitz continuous on the closed convex set , for any initial condition , there exists a unique Carathèodory solution to (4), , such that , for all , and that is uniformly continuous with respect to the initial condition.

Let be a compact sublevel set of the Lyapunov function that contains the initial condition . Then, the intersection set is a compact set. Let us consider the Carathèodory solution issuing from and evaluate the Lyapunov function along such solution. By (7

), and by a chain rule for absolutely continuous functions, for almost every

,

Thus, is non-increasing along , and we conclude that is a forward invariant set. Using a Gronwall–Bellman inequality, see the final arguments in the proof of [37, Th. 2], we obtain that

and therefore

(22)

which proves global exponential convergence.

References

  • [1] N. S. Kukushkin, “Best response dynamics in finite games with additive aggregation,” Games and Economic Behavior, vol. 48, no. 1, pp. 94–10, 2004.
  • [2] W. Saad, Z. Han, H. Poor, and T. Başar, “Game theoretic methods for the smart grid,” IEEE Signal Processing Magazine, pp. 86–105, 2012.
  • [3] F. Parise, M. Colombino, S. Grammatico, and J. Lygeros, “Mean field constrained charging policy for large populations of plug-in electric vehicles,” in Proc. of the IEEE Conference on Decision and Control, Los Angeles, California, USA, 2014, pp. 5101–5106.
  • [4] Z. Ma, S. Zou, L. Ran, X. Shi, and I. Hiskens, “Efficient decentralized coordination of large-scale plug-in electric vehicle charging,” Automatica, vol. 69, pp. 35–47, 2016.
  • [5] S. Grammatico, “Exponentially convergent decentralized charging control for large populations of plug-in electric vehicles,” in Proc. of the IEEE Conf. on Decision and Control, Las Vegas, USA, 2016.
  • [6] S. Grammatico, B. Gentile, F. Parise, and J. Lygeros, “A mean field control approach for demand side management of large populations of thermostatically controlled loads,” in Proc. of the IEEE European Control Conference, Linz, Austria, 2015.
  • [7] S. Li, W. Zhang, J. Lian, and K. Kalsi, “Market-based coordination of thermostatically controlled loads - Part I: A mechanism design formulation,” IEEE Trans. on Power Systems, vol. 31, no. 2, pp. 1170–1178, 2016.
  • [8] N. Li, L. Chen, and M. A. Dahleh, “Demand response using linear supply function bidding,” IEEE Trans. on Smart Grid, vol. 6, no. 4, pp. 1827–1838, 2015.
  • [9] J. Barrera and A. Garcia, “Dynamic incentives for congestion control,” IEEE Trans. on Automatic Control, vol. 60, no. 2, pp. 299–310, 2015.
  • [10] S. Grammatico, “Dynamic control of agents playing aggregative games with coupling constraints,” IEEE Trans. on Automatic Control, vol. 62, no. 9, pp. 4537 – 4548, 2017.
  • [11] F. Facchinei and J.-S. Pang, Finite-dimensional variational inequalities and complementarity problems.   Springer Science & Business Media, 2007.
  • [12] F. Facchinei and C. Kanzow, “Generalized nash equilibrium problems,” Annals of Operations Research, vol. 175, no. 1, pp. 177–211, 2010.
  • [13] D. P. Palomar and Y. C. Eldar, Convex optimization in signal processing and communications.   Cambridge university press, 2010.
  • [14] R. Cominetti, F. Facchinei, and J. Lasserre, Modern optimization modelling techniques.   Birkhäuser, 2010.
  • [15] L. Pavel, “An extension of duality to a game-theoretic framework,” Automatica, vol. 43, pp. 226 –237, 2007.
  • [16] H. Yin, U. Shanbhag, and P. Mehta, “Nash equilibrium problems with scaled congestion costs and shared constraints,” IEEE Trans. on Automatic Control, vol. 56, no. 7, pp. 1702–1708, 2011.
  • [17] A. A. Kulkarni and U. Shanbhag, “On the variational equilibrium as a refinement of the generalized Nash equilibrium,” Automatica, vol. 48, pp. 45 –55, 2012.
  • [18] F. Salehisadaghiani and L. Pavel, “Distributed Nash equilibrium seeking: A gossip-based algorithm,” Automatica, vol. 72, pp. 209–216, 2016.
  • [19] J. Koshal, A. Nedić, and U. Shanbhag, “Distributed algorithms for aggregative games on graphs,” Operations Research, vol. 64, no. 3, pp. 680–704, 2016.
  • [20] M. Ye and G. Hu, “Game design and analysis for price-based demand response: An aggregate game approach,” IEEE Trans. on Cybernetics, vol. 47, no. 3, pp. 720–730, 2016.
  • [21] ——, “Distributed Nash equilibrium seeking by a consensus based approach,” IEEE Trans. on Automatic Control, vol. 62, no. 9, pp. 4811–4818, 2017.
  • [22] S. Liang, P. Yi, and Y. Hong, “Distributed Nash equilibrium seeking for aggregative games with coupled constraints,” Automatica, vol. 85, pp. 179–185, 2017.
  • [23] S. Grammatico, F. Parise, M. Colombino, and J. Lygeros, “Decentralized convergence to Nash equilibria in constrained deterministic mean field control,” IEEE Trans. on Automatic Control, vol. 61, no. 11, pp. 3315–3329, 2016.
  • [24] S. Grammatico, “Aggregative control of large populations of noncooperative agents,” in Proc. of the IEEE Conf. on Decision and Control, Las Vegas, USA, 2016.
  • [25] G. Belgioioso and S. Grammatico, “Semi-decentralized Nash equilibrium seeking in aggregative games with coupling constraints and non-differentiable cost functions,” IEEE Control Systems Letters, vol. 1, no. 2, pp. 400–405, 2017.
  • [26] ——, “Projected-gradient methods for generalized equilibrium seeking in aggregative games are preconditioned forward-backward splittings,” in Proc. of the IEEE European Control Conference, 2018.
  • [27] A. Nagurney and D. Zhang, Projected dynamical systems and variational inequalities with applications.   Springer, 1996.
  • [28] R. Goebel, “Stability and robustness for saddle-point dynamics through monotone mappings,” Systems & Control Letters, vol. 108, pp. 16–22, 2017.
  • [29] M. González Vayá, S. Grammatico, G. Andersson, and J. Lygeros, “On the price of being selfish in large populations of plug-in electric vehicles,” Proc. of the IEEE Conf. on Decision and Control, pp. 6542–6547, 2015.
  • [30] L. Deori, K. Margellos, and M. Prandini, “On the connection between Nash equilibria and social optima in electric vehicle charging control games,” in Proc. of the IFAC World Congress, Toulouse, France, 2017.
  • [31] F. Facchinei and J. Pang, Finite-dimensional variational inequalities and complementarity problems.   Springer Verlag, 2003.
  • [32] G. Scutari, F. Facchinei, J.-S. Pang, and D. Palomar, “Real and complex monotone communication games,” IEEE Trans. on Information Theory, vol. 60, no. 7, pp. 4197–4231, 2014.
  • [33] G. Belgioioso and S. Grammatico, “On convexity and monotonicity in generalized aggregative games,” in Proc. of the IFAC World Congress, Toulouse, France, 2017.
  • [34] D. Gadjov and L. Pavel, “Continuous-time distributed dynamics for Nash equilibrium over networks via a passivity based control approach,” in Proc. of the IEEE Conf. on Decision and Control, 2017, pp. 4600–4605.
  • [35] A. Nagurney and D. Zhang, Projected Dynamical Systems and Variational Inequalities with Applications.
  • [36] A. Cherukuri, E. Mallada, and J. Cortés, “Asymptotic convergence of constrained primal-dual dynamics,” Systems & Control Letters, vol. 87, pp. 10 – 15, 2016.
  • [37] B. Brogliato and D. Goeleven, “The Krakovskii–LaSalle invariance principle for a class of unilateral dynamical systems,” Mathematics of Control, Signals, and Systems, vol. 17, no. 1, pp. 57 –76, 2005.