A rational decentralized generalized Nash equilibrium seeking for energy markets

06/04/2018 ∙ by Lorenzo Nespoli, et al. ∙ EPFL IDSIA SUPSI 0

We propose a method to design a decentralized energy market which guarantees individual rationality (IR) in expectation, in the presence of system-level grid constraints. We formulate the market as a welfare maximization problem subject to IR constraints, and we make use of Lagrangian duality to model the problem as a n-person non-cooperative game with a unique generalized Nash equilibrium (GNE). We provide a distributed algorithm which converges to the GNE. The convergence and properties of the algorithm are investigated by means of numerical simulations.



There are no comments yet.


page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

I-a Motivations

The scientific community agrees that in the future the intelligent activation of demand response (DR) will contribute to a reliable power system and price stability on power markets. Actuation of DR requires to solve an optimization problem in order to maximize an economic objective, which typically results in a welfare maximization problem (WMP), in which the unweighted sum of the economic costs of a group of agents is minimized. A very similar, and perhaps more studied problem, is the optimal power flow (OPF) problem. The OPF is usually solved in a centralized way by an independent system operator (ISO), in order to minimize the generation cost of a group of distributed power plants, over the set of underlying grid constraints. When the number of generators increases, the problem could become computationally expensive. Furthermore, retrieving all the generator-specific parameters could become impractical for the ISOs. For these reasons, different decentralized formulations of the OPF exist [1], which can speed up the computation exploiting parallelization among the different units. Furthermore, solving the problem in a decentralized way allows the generators to keep most of their information and parameters private, increasing privacy and lowering cyber-security concerns. The main difference between the OPF and DR setting, is that the second one involves the participation of self-serving agents, which cannot be a-priori trusted by the ISOs. This implies that if an agent find it profitable (in terms of its own economic utility), he will compute a different optimization problem from the one provided by the ISO. For this reason, some aspects of DR formulations are better described through a game theoretic framework.

I-B Background and previous work

In this setting, we must consider that agents can adopt a strategy , which can be in general different from the one suggested by the ISO, based on their private information (or type), denoted as , and their belief about the strategy of the other prosumers. The well-known Vickrey-Clarke-Groves (VCG) mechanism [2, 3, 4] belongs to the strategy-proof class of mechanisms and presents other useful theoretical properties, among which being weakly budget-balanced. Anyway, to achieve this, it requires a value redistribution among agents under the form of monetary taxation, such that the tax which applies to agent is directly or indirectly independent from its actions. This implies that optimization problems must be solved, each of which is performed without considering a given agent. This makes the computational cost quadratic in . Furthermore, VCGs are typically centralized and as such, they do not preserve the privacy of the agents. For example, in [5], a VCG mechanism for virtual inertia is considered, in which bidders send their bidding curves to a center, which solves independent optimization problems. Since the VCG mechanism guarantees that the best bidding strategy is bidding truthfully, they send their true cost curves to the center. Note anyway that, if the agent’s system presents some constraints, must represent them. This means that the center must know all the agent constraint sets in order to solve the VCG. The unfavorable computational cost makes the VCG impractical for combinatorial auctions [6] and problems with a large number of users with a nontrivial objective function. Despite this and other aspects which make it impractical in some cases [7], VCGs have been extensively studied since they are the only general purpose incentive compatible mechanisms which maximize social welfare [8]. In order to preserve agent’s privacy, it is possible to retrieve a distributed formulation of VCG using primal-dual decomposition algorithms. Note that distributing the mechanism aggravates the scalability problem of VCG, since the overall computation must now take into account communication delays. A second effect of adopting a decentralized formulation is that we cannot guarantee strategyproofness anymore. This is known as the cost of decentralization [9], which leads to a weaker notion of incentive compatibility, namely ex-post Nash equilibrium (EPNE). Although weaker than a dominant-strategy equilibrium, ex-post Nash equilibrium does not require agents to model the strategies nor types of other agents through belief functions, as it’s done using Bayes-Nash equilibrium [10]. Following this concept, in [11] guidelines for distributed implementations of VCG mechanisms are derived. In [9] a distributed VCG mechanism which reuses part of the computation done in each subproblem is presented. More recently [12] has proposed a distributed VCG implementation based on dual decomposition, and applied the concept of multistage mechanism, in which different mechanisms are applied at each primal dual update. Also in this case, the proposed algorithm scales quadratically with the number of agents. Another field of research, started with the seminal work of Rosen on n-person non-cooperative games [13], adopt non-VCG mechanisms to reach EPNE [14, 15]. This involves allowing a loss in terms of efficiency [16], with the benefit of better scalability with respect to the number of agents.

In this paper we propose a method to guarantee participation constraint, also known as individual rationality (IR): all the prosumers must have a positive return participating in the proposed energy market, with respect to the base case. We ensure IR allowing a coordinator to limit the Lagrangian multipliers associated to the coupling constraints. The rest of the paper is structured as follows: in II the specific problem we address is formulated and we show that its associate game mapping is monotone, which is a condition for the uniqueness of the VGNE; in III we propose a new algorithm for reaching the GNE, based on the alternating direction method of multipliers (ADMM); in IV we compare the convergence of the aforementioned algorithm with a recently proposed [17] preconditioned forward backward (pFB) algorithm for distributed Nash equilibrium seeking.

Ii Problem formulation

In this work we are interested in a more general problem with respect of the OPF. In particular, we consider the case in which a group of agents which produce and/or consume energy (prosumers now on) can sell their aggregated flexibility to third parties, for example to DSO through demand response programs or to balance responsible parties. The mathematical formulation of this problem is known as the sharing problem:


where is the Cartesian product of the prosumers feasible sets, is a system level objective, are the costs of each prosumers and the linear constraints are affine coupling constraints between the prosumers and

is the vector of the concatenated actions of all the prosumers. Here the affine coupling constraints encode grid constraints, limiting voltage and power in a subset of selected nodes of the grid in which the agents are located. This is possible taking into account the linearized formulation of the power flow equations

[1, 18]

, whose coefficients can be estimated using phasor measurement units

[19], even using smart meter data [20]. The advantage of considering coupling constraints instead of agent-level constraints on voltage and power is given by the fact that the first approach can reach better solutions in terms of total welfare.

As anticipated in the introduction, we are interested in decomposing problem (1) among the self interested prosumers, in such a way that the induced game presents only one variational GNE, and in the algorithms leading to such an equilibrium. Being the equilibrium unique, rational agents will converge to the EPGNE. This is equivalent to assume that the agents believe their own influence on the prices broadcasted by the sequence of mechanism proposed by the algorithm are negligible, i.e. they are price takers.

A reasonable way to turn the centralized problem (1) into a non-cooperative game, is to reward each prosumer with a part of the system level objective , based on the amount of energy he produces or consumes during a give period of time:


Anyway, this would result in a non-linear and non-convex game. As a first approximation we can replace this repartition rule with fixed (during each horizon) coefficients, based on a moving average:




Note that the game induced by the value functions in (2) defines an aggregative game [21], in which the each prosumer influence other’s prosumers value only by means of the aggregated actions. The induced game can be described as the set of optimization problems (5) in which each prosumer minimizes its own value function and associated KKT conditions (6).


where and is the normal cone operator. Before introducing the algorithms that can be used to solve 5, we discuss some properties of the proposed objective function. It is known that a sufficient condition for the existence and uniqueness of a NE for a n-person non cooperative game is that the system-level objective function is diagonally strictly convex [13]. In the case of affine coupling constraints, authors in [22] and [23] have shown that the game has a unique variational GNE if the pseudogradient of , also known as game mapping, is strictly monotone. Furthermore, the equilibrium can be reached making the agents pay , that is, the value function of each agent coincides with the integral of the first row of KKT in (6), . In this case, it has been shown that the agents reach a variational GNE with unique Lagrangian multiplier . Note that the game mapping differs from gradient of since its components are the partial derivatives of the values of the ith agent with respect to its own actions. We now show that the game map generated by the agents’ values defined in (2) inherits monotonicity from the convexity of .

Theorem II.1.

Let be a (strictly/strongly) convex function and let the costs of the agents be convex functions. Then any repartition of among the agents such that:

generates a (strictly/strongly) monotone game map


can be seen as a sum of two operators: and . Due to the separability of , it coincides with the gradient of . Due to the convexity of , is a monotone map, since the gradient of a convex function is monotone (theorem 1 in [24]). Using the same reasoning, is a monotone map due to the convexity of . From the definition of monotonicity, . Additionally, since any convex function must be convex along any path, we can state it component-wise: . Since we defined all as positive, . Thus , and is monotone being the sum of two monotone operators. ∎

Iii Algorithms for GNE seeking

As demonstrated in [23], asysmmetric projection algorithms [25] can be used to reach a GNE of an aggregative game with quadratic utilities. Recently, the same algorithm has been rigorously derived modeling the GNE as a monotone inclusion [17], showing that it coincides with a preconditioned forward backward (pFB) method (algorithm (1)), which is a special case of the Banach-Picard iteration [26] of two operators whose sum is the set value mapping associated to the KKT conditions in (6).

Algorithm 1 pFB

We compare algorithm (1) with a trivial modification of the ADMM algorithm [27], which convergence rate and properties have been extensively studied in the literature. For clarity of exposition, we start considering the version of problem (1) without coupling constraints. This can be solved in a centralized way through ADMM, applying the procedure in [27] §7.3, which results in the following parallelized formulation:

Algorithm 2 ADMM

where the only difference form the centralized algorithm is the coefficient in the update. We can write the KKT conditions at convergence


We can find from 12b and substitute it in 12a:


then we can use 12c, and recalling that is the summation matrix, we obtain:


which, together with 12d are equivalent to the KKT 6 of the game 5, when .

Iii-a Pricing and individual rationality

In this paper we only consider the case in which the function is the surplus that the agent community has in paying the energy at the point of common coupling with the electrical grid:


where is the vector of total power of the ith agent, is the energy cost function defined as:


where and are the buying and selling tariffs, respectively, at time . In order to induce agents to follow the proposed mechanism, we must ensure that the energy tariff they pay participating in the market is always lower than the one they pay in the base case. This is always true when we are not taking into account grid constraints, since as defined in (15) is always non-negative, when , as usual in energy tariffs. However, if the agents are located in a grid with big voltage oscillations, the Lagrangian dual variables (which we can interpret as punishment prices) could be such that the cost paid by the agents is higher than . To ensure IR, we encode it in the optimization scheme. At each iteration, for each time step in the horizon, we increment the Lagrangians only if the following condition holds:


where a negative value means that the prosumer is gaining a reward. This obviously results in the impossibility to satisfy the coupling constraints. We can give the following straightforward economic interpretation to this mechanism: each agent would opt-out from the game as soon as the energy tariffs become unfavorable with respect to the existing one. Condition (17) prevent this from happening. In the presence of bad power quality, the DSO could provide favorable energy tariffs to prosumers participating in the mechanism, ensuring that condition (17

) is met with high probability.

Iii-B Prosumers problem formulation

In this paper, each prosumer’s flexibility is modeled using an electric battery. Although simple, the model we used is not simplistic and we briefly describe it in this subsection. Since the effect of charging or discharging on the state of charge is not symmetric due to the efficiencies, the problem is usually formulated as a mixed integer linear program (MILP), introducing binary decision variables and using bilinear constraints, to avoid the simultaneous charge and discharge of the battery. Furthermore, the objective function of the agents is non differentiable at

and is mathematically described by the maximum operator. In order to speed up the computations, we reformulated all the control problems as a quadratic optimization.

We start considering that both the ADMM and the pFB formulations can be described by the following optimization problem:


where is a reference signal and , performs the sum of the charging and discharging operations with appropriate signs. Here, with abuse of notation, we redefined the vector as the vector containing both the charging and discharging operators, in such a way that , where and are the charging and discharging powers of the battery. For the ADMM formulation, it is easy to see that problem (18) can be used to solve (7). We can still use (18) for solving the pFB formulation recalling that the projected gradient descent is equivalent to a quadratic optimization problem in the form:


The battery is modeled as a discrete linear system, with the state of charge denoted by :


We can eliminate the dependence on the state of the optimization problem, using the standard batch formulation:


where is the control vector for the whole time horizon and are the batch matrices.

We can now describe the set through the linear constraints , defined as:


where are the power and energy box constraints, while

is the identity matrix of appropriate dimensions. Now we can reformulate the non differentiable cost function (

16) with a linear function, such as we can reuse it in both the ADMM and pFB formulations. We start considering that the if condition of the cost function in (16) can be equivalently formulated using the max operator. In turn, the max operator can be replaced by the sum of an auxiliary variable and appropriate inequality constraints. We augment our decision variable such as . Now the minimization of (16) is equal to the following optimization problem:


where and , and


where the rows entries are identical to the buying and selling prices at time . The effect of the matrices in (24) is that the new auxiliary variable is now an upper envelope for the cost function (16). Since we require the cost to be minimized, will coincide with the cost function at optimality. We can now use and in both the ADMM and pFB formulations. While in the first replaces in (18), in the latter the prosumers’ energy costs are considered as part of the pseudogradient: .

This problem formulation prevent us from introducing binary variables for the charging and discharging powers, since optimal solutions of (

23) does not require to simultaneously charge and discharge the battery at the same time. This is not true for the ADMM formulation, in which the quadratic penalty on the sum of and with respect to a reference signal is present. In the case in which the reference signal is negative, the battery is not only incentivized to charge itself, but to consume as much energy as possible. This will result in a simultaneous charge and discharge, due to the round-trip efficiency. To avoid this behavior, can be augmented with a linear therm, punishing the battery discharging operation when is negative: where has non zero entries, all identical to a punishment therms, only when .

Iv Numerical analysis

We test both the ADMM and the pFB and compare the performance to a centralized solution. The only difference from the ADMM and the centralized formulation are the coefficients in equation (7), which are not present in the centralized solution. Since the system-level objective function , as defined in (15), is not differentiable in 0 and is not strictly nor strongly convex, the convergence of pFB is not guaranteed. To have an equal comparison, we replaced the system-level cost function (16) with a continuously differentiable function. We define it by means of its derivative:


where regulates the steepness of the function in .

Fig. 1: Time series example, N = 10. Blue: forecasted profiles. Red: constraints. Grays: solutions of the centralized and decentralized approaches. Top: state of charge for each battery. Middle: power profiles. Bottom: voltage profiles.

In our simulations , which provides a reasonable steepness for all the possible values of the power aggregate, since we did all the computations in per units, and the aggregate power constraint is . We stress out that this approximation is only used for the system-level objective, and not for the prosumers objective functions, where the cost (16) is modeled as described in subsection III-B. In order to fairly compare the algorithms, we used an equal stepsize

, fixed to 0.1. The power profiles of each prosumer are randomly chosen from a yearly dataset of real residential electrical consumption. Each prosumer is equipped with a PV field, with a nominal power uniformly distributed between 2 and 10 times its daily energy consumption. Furthermore, each prosumer is provided with an electric battery with size equal to the expected daily energy exceeding its consumption. Figure

1 shows the optimized time series from a single case. In the upper panel, the batteries’ state of charge (SOC) are shown. Since the SOC is the time integral of the optimization variables (), it is clear that the ADMM and the pFB converged exactly to the same solution. The middle panels shows the forecasted aggregated power profile and the optimized one. Note that both the ADMM and pFB solutions are not far from the centralized solution, while differences are more evident in terms of single prosumers SOC. The last panel shows voltage profiles at the point of common coupling. In figure 2 the convergence of the two algorithms is shown, in terms of game objective function . We ran a total of 50 simulations, each of which includes 10 prosumers with power profiles and battery sizes randomly chosen, as explained before. For each simulation , we retrieve the best optimal value of , , defined as:


where are the solution of the two algorithms after 200 iterations (after which the relative change in for all the simulations was smaller than ). The thick lines show the median, while the shadowed patches contain half of the simulations.

Fig. 2: Normalized optimal value . The thick lines denote the median, while the shaded areas are the and quantiles.

V Conclusions

We have proposed a method to enforce IR while reaching a EPGNE in a distributed way. The method and the related algorithm have been tested, and compared with pFB, a state of the art algorithm for GNE seeking. The simulations shows that the proposed algorithm reaches the same solutions of pFB, while showing faster convergence in most of the cases. In future research, we will extensively investigate the advantages of the proposed methodology.


  • [1] D. K. Molzahn, F. Dorfler, H. Sandberg, S. H. Low, S. Chakrabarti, R. Baldick, and J. Lavaei, “A Survey of Distributed Optimization and Control Algorithms for Electric Power Systems,” IEEE Transactions on Smart Grid, vol. 3053, no. c, pp. 1–1, 2017.
  • [2] L. Makowski and J. M. Ostroy, “Vickrey-Clarke-Groves mechanisms and perfect competition,” Journal of Economic Theory, vol. 42, no. 2, pp. 244–261, 1987.
  • [3] E. H. Clarke, “Multipart pricing of public goods,” Public Choice, vol. 11, no. 1, pp. 17–33, 1971.
  • [4] W. Vickrey, “Counterspeculation, auctions, and competitive sealed tenders.,” The Journal of Finance, vol. 16, no. 1, pp. 8–37, 1961.
  • [5] B. K. Poolla, S. Bolognani, L. Na, and F. Dorfler, “A Market Mechanism for Virtual Inertia,” arXiv, 2017.
  • [6] V. Conitzer and T. Sandholm, “Failures of the VCG mechanism in combinatorial auctions and exchanges,” The 5th International Joint Conference on Autonomous and Multiagent Systems, pp. 521–528, 2006.
  • [7] M. H. Rothkopf, “Thirteen Reasons Why the Vickrey-Clarke-Groves Process Is Not Practical,” Operations Research, vol. 55, no. 2, pp. 191–197, 2007.
  • [8] N. Noam, T. Roughgarden, E. Tardos, and T. Wexler,

    Algorithmic Game Theory

    .   Cambridge University Press, 2007.
  • [9] A. Petcu, B. Faltings, and D. C. Parkes, “M-DPOP: Faithful distributed implementation of efficient social choice problems,”

    Journal of Artificial Intelligence Research

    , vol. 32, pp. 705–755, 2008.
  • [10] Y. Narahari, Game Theory and Mechanism Design, 2014, vol. 4.
  • [11] D. C. Parkes and J. Shneidman, “Distributed Implementations of Vickrey-Clarke-Groves Mechanisms,” Third Joint Conference on Autonomous and Multiagent Systems, pp. 261–268, 2004.
  • [12] T. Tanaka, F. Farokhi, and C. Langbort, “Faithful implementations of distributed algorithms and control laws,” IEEE Transactions on Control of Network Systems, vol. 4, no. 2, pp. 191–201, 2017.
  • [13] J. . B. . Rosen, “Existence and Uniqueness of Equilibrium Points for Concave N-Person Games,” Econometrica, vol. 33, no. 3, pp. 520–534, 1965.
  • [14] B.-g. Kim, S. Member, S. Ren, M. V. D. Schaar, J.-w. Lee, and S. Member, “Bidirectional Energy Trading and Residential Load Scheduling with Electric Vehicles in the Smart Grid,” IEEE Journal on selected areas in communication, vol. 31, no. 7, pp. 1219–1234, 2013.
  • [15] B. Gharesifard, T. Başar, and A. D. Domínguez-García, “Price-based coordinated aggregation of networked distributed energy resources,” IEEE Transactions on Automatic Control, vol. 61, no. 10, pp. 2936–2946, 2016.
  • [16] B. Yang and M. Johansson, “Distributed optimization and games: A tutorial overview,” in Networked Control Systems, 2010, vol. 406, pp. 109–148.
  • [17] G. Belgioioso and S. Grammatico, “Projected-gradient algorithms for generalized equilibrium seeking in Aggregative Games are preconditioned Forward-Backward methods,” 2018. [Online]. Available: http://arxiv.org/abs/1803.10441
  • [18] H. Almasalma, J. Engels, and G. Deconinck, “Dual-decomposition-based peer-to-peer voltage control for distribution networks,” CIRED - Open Access Proceedings Journal, no. 1, pp. 1718–1721, 2017.
  • [19] C. Mugnier, K. Christakou, J. Jaton, M. De Vivo, M. Carpita, and M. Paolone, “Model-less/measurement-based computation of voltage sensitivities in unbalanced electrical distribution networks,” 19th Power Systems Computation Conference, PSCC 2016, 2016.
  • [20] S. Weckx, S. Member, R. D. Hulst, J. Driesen, and S. Member, “Voltage Sensitivity Analysis of a Laboratory Distribution Grid With Incomplete Data,” IEEE Transactions on Smart Grid, vol. 6, no. 3, pp. 1271–1280, 2015.
  • [21] M. K. Jensen, “Aggregative games and best-reply potentials,” Economic Theory, vol. 43, no. 1, pp. 45–66, 2010.
  • [22] G. Belgioioso and S. Grammatico, “Semi-Decentralized Nash Equilibrium Seeking in Aggregative Games With Separable Coupling Constraints and Non-Differentiable Cost Functions,” IEEE Control Systems Letters, vol. 1, no. 2, pp. 400–405, 2017.
  • [23] D. Paccagnan, M. Kamgarpour, B. Gentile, F. Parise, J. Lygeros, and D. Paccagnan, “Distributed computation of Nash Equilibria in aggregative games with coupling constraints,” in 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, USA, 2016, pp. 6123–6128.
  • [24] G. J. Minty, “On the Monotonicity of the Gradient of a Convex Function,” Pacific J. Math., vol. 14, no. 1, pp. 243–248, 1963.
  • [25] F. Facchinei and J. S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, 2015, vol. 1542, no. 9.
  • [26] H. H. Bauschke and P. L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces, 2011.
  • [27] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers,”

    Foundations and Trends® in Machine Learning

    , vol. 3, no. 1, pp. 1–122, 2010.