One key enabler of smart grid energy trading and management schemes is the presence of prosumers
, i.e., smart grid customers capable of generating and storing their own energy. Indeed, the notable increase in penetration of solar photovoltaic (PV) panels and storage devices at the prosumer side of the grid will lead to novel demand-side energy management (DSM) schemes that will help alleviate the extremely expensive peak consumption hours and match consumption demand to the intermittent renewable energy supply of the grid.
A number of recent works [2, 3, 4] have studied the role of storage devices in grid energy management. For example, the authors in  propose a storage scheduling and management model, with the goal of minimizing peak hour consumption. In , consumer-based storage scheduling is proposed in an attempt to match energy consumption to the expected power output of a central wind generation unit. In addition, the work in  studies the topic of optimal offerings for wind power generators, through the application of a common storage device. Moreover, the work in  adopts a stochastic optimization and robust optimization approach to study demand response for residential appliances under uncertain real-time prices.
Game-theoretic methods have been widely applied in the existing DSM literature, given the coupled interactions between prosumers as discussed in [4, 6, 7, 8, 9, 10]. For instance, the authors in  propose a load scheduling technique with a dynamic pricing strategy related to the total consumption of the grid. More particularly, a number of works [7, 8, 9, 10] have used the framework of Stackelberg games in order to study the hierarchical interactions between the power company and the grid’s consumers. For example, the authors in  propose a Stackelberg game approach to deal with demand response scheduling under load uncertainty based on real-time pricing in a residential grid. Similarly, the authors in  and  use a Stackelberg game approach between one power company and multiple users, competing to maximize their profits, with the goal of flattening the aggregate load curve. On the other hand, the authors in  propose a Stackelberg game approach between company and consumers, while studying the impact that a malicious attacker could have, through the manipulation of pricing data. In addition, the works in  and  have used a Stackelberg game approach to characterize the demand response of consumers with respect to the retail price. In particular, the authors adopted stochastic and robust optimization methods to study energy trading with uncertain market prices. Moreover, a Stackelberg game for energy sharing management of microgrids with photovoltaic prosumers has been proposed in .
The main drawback of these works [4, 6, 7, 8, 9, 10, 11, 12, 13] is the assumption that consumers are fully rational and will thus choose their strategy in accordance to classical game theory (CGT). In practice, as observed by behavioral experimental studies , human players can significantly deviate from the rational principles of CGT, when faced with the uncertainty of probabilistic outcomes. In this regard, the framework of prospect theory (PT)  has been extensively applied to model the irrational behavior of real-life individuals in an uncertain decision making scenario . In fact, the authors in  and  discuss a storage management framework where the owner can choose to store or sell energy, while accounting for its subjective perceptions, using a PT framework. In , a PT framework is used for DSM to identify optimal customer participation time. However, these works [16, 18, 19] do not typically study the conflicting hierarchical interaction between the prosumers and the power company and instead focus on the consumer side of the grid. These works also fail to account for the uncertainty associated with variable or dynamic pricing, which is expected to play a major role in DSM . Even though the combination of Stackelberg games and PT has been applied in other research fields including wireless communication [20, 21], security games [22, 23], and transport theory , such combination has not been addressed in demand-side energy management problems and from an algorithmic perspective. In particular, these earlier works mainly focus on the weighting effect of PT while here, we consider the framing effect.
The main contribution of this paper is a novel hierarchical framework for optimizing energy trading between prosumers and a grid power company, while explicitly taking into account the uncertainty of the future energy price. Our work differs from most of the existing literature on energy trading [2, 3, 5, 4, 6, 7, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 20, 21, 22, 23, 24] in several aspects: 1) It models the behavior of prosumers who can both generate and consume energy under price uncertainty and using a Stackelberg game, 2) It provides a simple distributed algorithm with polynomial convergence rate to the unique Stackelberg equilibrium point, and 3) It captures the subjective decision making behavior of the prosumers using framing effect in PT.
In particular, we formulate a single-leader, multiple-follower Stackelberg game, in which the power company, acts as a leader who declares its pricing strategy in order to maximize its profits, to which prosumers, acting as followers, react by choosing their optimal energy bid. We define a prosumer’s utility function that captures the profits resulting from buying/selling energy at the current known price, as well as the uncertain future profits, originating from selling stored energy. In contrast to CGT, we develop a PT framework that models the behavior of prosumers when faced with the uncertainty of future profits. In particular, we account for each prosumer’s valuation of its gains and losses, compared to its own individual utility evaluation perspective, as captured via PT’s framing effect , by introducing a utility reference point. This reference point represents a prosumer’s anticipated profits and originates from previous energy trading transactions and future aspirations of profits, which can differ in between prosumers . We show that, under CGT, the followers’ noncooperative game admits a unique pure strategy Nash equilibrium (NE). Moreover, under PT, we derive a set of conditions under which the pure strategy NE is shown to exist. In particular, we propose distributed algorithms that allow the prosumers and power company to reach an equilibrium under both CGT and PT. Simulation results show that the total grid load, under PT, decreases for certain ranges of prosumers’ reference points and increases for others, when compared to CGT. The results also highlight the impact of this variation on the power company’s profits, which significantly decrease, when it fails to account for the prosumers’ subjective perceptions under PT.
The rest of this paper is organized as follows. Section II presents the system model and formulates the Stackelberg game model. In section III, we present the game solution under CGT and provide a distributed algorithm which can quickly reach the Stackelberg solution of the game. We extend our results to games under PT in Section IV. Section V presents our simulation results, and finally conclusions are drawn in Section VI.
Ii System model
Consider the set of grid prosumers. Each prosumer owns an energy storage unit of capacity , and a solar PV panel which produces a daily amount of energy . Each prosumer has a known load profile that must be satisfied and an initial stored energy available in a storage device, originating from an excess of energy at a previous time.
In our model, the power company requires prosumers to declare the amount of energy that they will be buying or selling at the start of the period as done in day-ahead scheduling models used in DSM literature such as in  and . We let be the amount of energy declared by prosumer , where implies an amount of energy that will be bought and will represent the amount of energy that will be sold. indicates that no energy is traded.
The price of selling or buying one unit of energy is related to the total energy declared by all prosumers. In our pricing model, each prosumer is billed based on the amount that is declared. We assume that the prosumers are truthful and have no incentive to deviate, given the possible penalties that will be incurred. Next, as done in [18, 25], and , we choose a so-called fairness pricing for buying/selling energy which is proportional to the prosumers’ aggregate demand and given by:
where and are design parameters set by the power company. For simplicity, we assume that is fixed and positive, and that the company only varies to control the amount of energy bough/sold by the prosumers.111Note that in this function we can also allow the utility company to adjust . While this does change our analysis from the prosumers’ perspective, however, it gives an extra freedom for the utility company to maximize its utility at a cost of solving a more complex optimization problem. In (1),
is a vector that represents the amount of energy declared by all the prosumers in the set. The price of unit of energy is regulated and must be within a range . Here, we assume that the structure of this pricing function is pre-determined by the utility company and announced a priori to all the prosumers. This function is chosen based on the idea that a higher aggregate demand by the prosumers must naturally increase the energy prices.
The future price of energy is perceived to be unknown by the prosumers, given the uncertainty related to future solar energy generation and the pricing strategy of the power company. The future price of energy is thus modeled by a random variable. For simplicity, we assume that
follows a uniform distribution. However, most of our analysis can be extended to the case in which follows more general distributions.
The set of possible values of for each prosumer is . is a prosumer’s maximum sold/minimum bought energy. , is the maximum energy that prosumer can purchase. For a chosen energy bid , the prosumer’s utility function will be:
In (2), the first term represents the revenue/cost of prosumer at the current time, while the second term represents the future monetary value associated with unsold energy. In particular, is the amount of energy that prosumer will have in its storage in the future. The prosumers’ actions are coupled through the energy price and they will thus be competing to maximize their respective revenues. On the other hand, the power company will purchase (sell) the energy bough (sold) by the prosumers in the energy market at the current market clearing price . Given the current market price, the power company’s utility function is given by:
where the first term represents the revenue that the utility company earns by selling (buying) energy units to prosumers at the price of , while the second term is the cost of purchasing energy units at the clearing price of from the energy market.
The power company’s revenues are clearly affected by the prosumers and their energy bids. On the other hand, since the prosumers react to the power company’s choice of , the prosumers’ utility is directly affected by the power company’s action. We thus model the energy trading problem as a hierarchical Stackelberg game  with the power company acting as leader, and the prosumers acting as followers.
Ii-a Stackelberg game formulation
We formulate a single-leader, multiple-follower Stackelberg game , between the power company and the prosumers. The power company (leader), will act first by choosing to maximize its profits. The prosumers, having received the power company’s pricing strategy, will engage in a noncooperative game. In fact, the final price of energy is proportional to the grid’s total load, to which each prosumer contributes. We first formulate the prosumers’ problem under CGT as follows:
In (4), prosumer attempts to maximize its expected profits, given the actions of other prosumers and the power company. The previous formulation assumes all prosumers to be rational expected utility maximizers. Moreover, the power company’s problem will be:
To solve this game, one suitable concept is that of a Stackelberg equilibrium (SE) as the game-theoretic solution of our model.
A strategy profile () is a Stackelberg equilibrium if it satisfies the following conditions:
where is the solution to problem for all prosumers in , and is the solution to problem .
It is worth noting that our Stackelberg formulation is based on a static game. However, our solution to this problem is based on the notion of a repeated game approach in which the prosumers frequently interact with the utility company in order to find their equilibrium strategies. This is practically important as it is a step toward analyzing more complex senarios in which the smart grid’s environment dynamically changes from one time instant to the other. For instance, one can consider a multi-stage game where for each prosumer , and dynamically vary with time , while is affected by , and action taken at previous and current times. As such, to capture this dynamic nature, our proposed static game model can be expanded to a dynamic stochastic game [28, 29, 30] with transition equations describing the evolution of the states, corresponding to , , and , with respect to time depending on the control inputs and previous states. In this case, the state of the game at time consists of , , and for prosumer at stage . The chosen optimal action by each player at time , i.e. control inputs, would depend on the state at which the game is. In this respect, our static game analysis here provides analytical and algorithmic solution approaches through which optimal strategies, for the prosumers and energy company, can be obtained at the stationary states of the stochastic game.222In a stochastic game framework, a stationary strategy consists of obtaining the optimal strategy for a certain state regardless of the history of the game or the particular time instant at which the game is played. In other words, our repeated single stage game analysis can be viewed as a solution to the multi-stage stochastic game which has reached its stationary condition (i.e., at the stationary state, it appears as if one is repeatedly playing the same stationary game). In particular, the optimal stationary strategies can be extracted from our static game analysis under the stationarity condition.
Iii Game Solution under CGT
The analysis under CGT assumes that all prosumers are expected utility maximizers. Thus, we seek to find a solution that solves both problems and , while satisfying (1). First, we start by solving the follower’s problem while assuming the the leader’s action is fixed to . We now introduce the following notations:
Here, the expected utility of prosumer will be:
Next, we denote by the best response of player , which is the solution of problem , given that all the other players choose a specific strategy profile . The following theorem explicitly characterizes the best response of each prosumer .
The best response of player is given by:
See Appendix A. ∎
In fact, one can rewrite the best responses of all the players in Theorem 1 in a combined single matrix form. We define to be an matrix with all entries equal to - except the diagonal entries which are , i.e., if , and , otherwise. Let where is the vector of all ’s. Then, we can rewrite (9) for all players as
where is the projection operator on the dimensional cube in . Our analysis will later use this closed-form representation of the best response dynamics.
Iii-a Existence and uniqueness of the followers’ NE under CGT
One key question with regard to the prosumers’ game is whether such a game admits a pure-strategy NE. This is important as it allows us to stabilize the demand market in an equilibrium where each prosumer is satisfied with its payoff, as shown next.
The prosumers’ game admits a unique pure-strategy NE.
See Appendix B. ∎
Iii-B Distributed learning of the followers NE
Next, we propose a distributed learning algorithm which converges in a polynomial rate to the unique pure-strategy NE of the prosumers’ game as formally stated in Algorithm 1. At each stage of Algorithm 1, prosumer selects its next action as a convex combination of its current action and its best response at that stage. One of the main advantages of Algorithm 1 is that it can be implemented in a completely distributed manner as each prosumer needs only to know its own actions and best response function, and does not require any information about others’ actions. Moreover, the prosumers do not need to keep track of their actions history which is the case in many other learning algorithms. Note that Algorithm 1 can be viewed as a special case of more general algorithms known as relaxation algorithms .
The idea behind Algorithm 1 is that each player initially puts more weight on its best response in order to explore faster other possible actions with better payoffs. As the time elapses, the prosumers’ actions become closer to their optimal actions and, hence, the prosumers exploit their current actions by putting more weight on their own actions. While the exploration coefficient can be replaced by other possible coefficients, we have chosen to optimize the speed of convergence. Finally, note that the implementation of Algorithm 1 is made possible by a bidirectional communication between the power company and the prosumers, provided by the smart meters. In fact, at each iteration, the prosumer would send the power company its current strategy, and would receive the updated energy price.
Next, we consider the following definition which will be handy in proving our main convergence result in this section:
Given an players game with utility functions and any two action profiles and , the Nikaido-Isoda function associated with this game is given by .
The Nikaido-Isoda function measures the social income due to selfish deviation of individuals. This function admits several key properties. As an example we always have . Moreover, given a fixed action profile , is maximized when equals the best response of player with respect to . In particular, for such a best response action profile , if and only if is a pure strategy NE of the game. While the Nikaido-Isoda function has been used earlier to prove convergence of certain dynamics to their equilibrium points [32, 31], however it usually fails to provide an explicit convergence rate. In the following theorem we leverage the Nikaido-Isoda function associated to the prosumers’ game to measure the distance of outputs of Algorithm 1 from the Nash equilibrium, and hence obtain an explicit bound on the convergence rate of this algorithm.
If every prosumer updates its energy request bid based on Algorithm 1, then their action profiles will jointly converge to an pure strategy NE. After steps the joint actions will be an -NE where (i.e., the convergence rate to an NE is ).
See Appendix D. ∎
As it has been shown in Appendix B, the prosumers’ game is a concave game , which is known to admit a distributed learning algorithm for obtaining its NE points (see, e.g., [33, Theorem 10]). However, in general obtaining distributed learning algorithms with provably fast convergence rates to NE points in concave games is a challenging task. Therefore, one of the main advantages of Theorem 3 is that it establishes a polynomial convergence rate for the relaxation Algorithm 1 leveraging rich structure of the prosumers’ utility functions.
Iii-C Finding the Stackelberg Nash equilibrium under CGT
While Algorithm 1 achieves a unique pure strategy for the prosumers’ game under CGT, our final goal is to obtain the Stackelberg equilibrium of the entire game. For this purpose, we leverage Algorithm 1 using one of the following methods to construct the SE of the entire market under CGT:
Iii-C1 Method 1
The Stackelberg equilibrium of the game can be found by solving the following non-linear optimization problem. Let be the unique NE obtained by the followers when the power company’s action is . Note that is a well-defined continuous function of . First, the power company solves the following optimization problem a priori to find its unique optimal action and announces it to the prosumers. The problem is defined as:
In fact, one can characterize the unique pure-strategy NE of the prosumers’ game given in (11) in more detail. Since at equilibrium every player must play its best response in (10), therefore, an action profile is an equilibrium if and only if we have , which means . Since the former is a convex optimization problem, we can write its dual as
where and are the dual variables corresponding to the constraints and , respectively. We denote the optimal solution of the dual by . Since, we already know that is the optimal solution of the primal, due to the strong duality the values of the primal and dual must be the same, i.e., . Moreover, the KKT conditions must hold at the optimal solution , which, together with provide the following system of equations with variables which characterizes the equilibrium point using dual variables:
Solving these equations can be used to derive the unique pure-strategy NE of the followers’ game.
We next present a second method, which does not require the power company to solve the non-linear inequalities in (14). In addition, the second method allows the players to reach the SE quickly and efficiently in steps and will be mainly used in our simulation results in Section V.
Iii-C2 Method 2
Given any small for which the power company and the prosumers want to find their -SE with precision (i.e., no one can gain more than by deviating), the company partitions its action interval and sequentially announces prices . For each such price , prosumers obtain their -equilibrium in no more than steps, and the company must repeat this process at most steps and choose the action that maximized its utility. The running time in this case will be to find an -SE.
Our analysis thus far assumed that all prosumers are fully rational and their behavior can thus be modeled using CGT. However, this assumption might not hold , given that prosumer are humans that can have different subjective valuations on their uncertain energy trading payoffs. Next, we extend our result using PT  to model the behavior of prosumers when faced with the unknown future price of energy and thus the actual value of the stored energy.
Iv Prospect theoretic analysis
In a classical game-theoretic framework, an individual evaluates an objective expected utility. However, in a real-life setting, empirical studies , have shown that decision makers, tend to deviate noticeably from the rationality axioms, when subjected to uncertain payoffs. The most prominent of such studies was that done by Kahneman and Tversky within the context of prospect theory , which won the 2002 Nobel prize in economic sciences.
The utility framing notion is one of the two main tenets of prospect theory. As observed in real-life experimental studies, utility framing states that each individual perceives a utility as either a loss or a gain, after comparing it to its individual reference point . The reference point is typically different for each individual and originates from its past experiences and future aspirations of profits. Furthermore, individuals tend to evaluate losses in a very different manner compared to gains. The main axioms of utility framing are summarized as follows:
Individuals perceive utility according to changes in value with respect to a reference point rather than an absolute value.
Individuals assign a higher value to differences between small gains or losses close to the reference point in comparison to those further away. Te effect is referred to as diminishing sensitivity, and is captured by the coefficients and .
Individuals feel greater aggravation for losing a sum of money than satisfaction associated with gaining the same amount of money. This phenomenon is referred to as loss aversion and is captured by the aversion coefficient .
It is worth noting that PT differs from other risk measures such as Conditional Value at Risk (CVaR)  which evaluates the market risk based on the expected value of the risk at some future time. The underlying assumption in evaluating CVaR is that the risk is measured based on the conventional expectation of the future uncertain price, while in PT, the expectation is replace by subjective perception of the individuals which up to some extent introduces a notion of bounded rationality into the model.
Iv-a Energy Trading Analysis through Utility Framing
In our model, a prosumer’s uncertainty originates from the unknown future energy price and power company pricing strategy. Consequently, we will analyze the effect of the key notion of utility framing from PT. Utility framing states that a utility is considered a gain if it is larger than the reference point, while it is perceived as a loss if it is smaller than that reference point. This reference captures a prosumer’s anticipated profits and originates from past energy trading transactions and future aspirations of profits, which can differ in between different prosumers . Let be the reference point of a given prosumer . Thus, to capture such subjective perceptions, we use PT framing  to redefine the utility function:
Iv-B Existence and uniqueness of the NE under PT
To study the existence of the followers’ NE under PT, we analyze the concavity of the utility function in (16). The concavity of the PT utility function provides a sufficient condition to conclude the existence of at least one pure-strategy NE [33, Theorem 1]. Here, we note that prosumer ’s expected utility function can take multiple forms over the product action space , depending on the conditions in (16). It is thus challenging to prove that the utility function is concave, which makes it extremely difficult to analyze the existence and uniqueness of the followers’ NE. Thus, we inspect a number of conditions under which the PT utility function is concave. Here, for simplicity and to provide more closed-form solutions, we disregard the diminishing sensitivity effect and thus set . The following theorem provides sufficient conditions under which the prosumers’ game under PT admits a pure NE.
In either of the following cases, the prosumers’ game under PT admits at least one pure strategy NE:
Case 1: .
Case 2: .
Case 3: , and or , and ,
See Appendix E. ∎
As an immediate corollary of Theorem 4, under any of the above conditions, one can again obtain the SE of the entire market for PT prosumers using the same procedure used under CGT (i.e., using Algorithm 1 in the prosumers’ side together with either of the methods in Subsection III-C). This is simply because, under any of the conditions in Theorem 4, the prosumers’ game again becomes a concave game which is sufficient for the convergence of Algorithm 1. It is worth noting that, in general, using PT rather than CGT will change the results pertaining to the existence of an NE (see e.g., ). However, Theorem 4 provides a sufficient condition under which the same existence results derived for CGT still hold under PT.
Finally, whenever the concavity of the game cannot be guaranteed, we propose a sequential best response algorithm, that build on our previous work in . This is a special case of Algorithm , where , and where players update their strategy sequentially instead of simultaneously. An analytical proof of existence/convergence is challenging, given that no proof for the game’s concavity could be derived, as previously discussed. However, when it converges, this algorithm is guaranteed to reach an NE. In fact, as observed from our simulations in Section V, the algorithm always converged and found a pure-strategy NE, for all simulated scenarios.
V Simulation Results and analysis
For our simulations, we consider a smart grid with prosumers, unless otherwise stated, each of which having a load arbitrarily chosen within the range kWh. In addition, the storage capacity is set to 25 kWh and . and are taken to be both equal to 0.88 and , unless stated otherwise . We set, and , unless stated otherwise. When the leader’s action is not fixed, method from Section III-C was used to find the SE.
Fig. 1 compares the effect of different prosumer reference points on the total energy sold or bought for both CGT and PT, while fixing the power company’s action. For CGT, a prosumer’s reference point is naturally irrelevant. For the PT case, for a reference point below , the prosumers’ action profile is not significantly affected compared to CGT, since most potential payoffs of the action profile are still viewed as gains, above the reference point. As the reference point increases from to , the total energy consumed will decrease from around kWh to kWh, since some of the potential payoffs of the current action profile will start to be perceived as losses, as they cross the reference point. Given that losses have a larger weight under PT compared to CGT, the expected utility of the current strategy profile will significantly decrease thus causing the followers to exhibit a risk-averse behavior. In fact, as some of the potential future profits are perceived as losses, a prosumer will sell more energy at the current time slot. As the reference point increases from to , the present profits are now perceived as losses, and prosumers will start exhibiting risk-seeking behavior. In fact, each prosumer will consider the present profit as insignificant and will thus store more energy in the hope of selling it in the future at higher prices. Finally, as the reference point approaches $8, the effect of uncertainty will gradually decrease, given that all profits are now perceived as losses. We note that even a small difference in perception () caused the total grid load to shift from kWh to kWh. This highlights the importance of behavioral analysis and prosumer subjectivity when assessing the performance of dynamic pricing strategies.
Fig. 2 shows the effect of the loss multiplier on the total energy purchased, for a fixed power company strategy. The loss multiplier maps the loss aversion of prosumers when assessing their utility outcomes. The effect of framing is more prominent as the loss multiplier increases. For instance, the prosumers will exhibit more risk averse behavior for a reference point in the range of . As seen from Fig. 2, as increase from to , the total load would decrease by up to . In fact, to avoid the large losses, the prosumers will decrease the energy they purchase at the current risk free energy price.
Fig. 3 compares the company’s profits for the scenario in which the power company accounts for prosumer irrationality to the scenario in which the power company assumes that prosumers are rational. In both scenarios, the prosumers are irrational. For a reference point below , the company’s profits are barely affected. However, as the reference point crosses , the company’s profits start to show a clear decrease between the two scenarios. In fact, as the power company is not accounting for the prosumer’s actual subjective behavior, its pricing strategy is no longer optimal. As was seen in Fig. 1, this is the reference point range where the total consumption mostly differs between CGT and PT. The decrease in profits reaches a peak value of 15 % at a reference point of $2. Clearly, the power company will experience a decrease in profits, if it neglects the subjective perception of prosumers.
Fig. 4 shows the total grid load energy consumption as function of the number of prosumers. The figure highlights the difference in consumption between rational prosumers and subjective prosumers with , which increases significantly with the number of prosumers in the grid. This difference reaches kWh for prosumers. This highlights the impact of irrational behavior, which is prominent for larger grids.
Fig. 5 shows the energy consumption of different groups of prosumers, with different reference points, inside a single grid. For a very small , the different groups have equal consumption. As is increased to cents, the prosumers with start to decrease their consumption at equilibrium, while the other groups’ consumption remains unchanged. This is similar to what was discussed in Fig 1, where prosumers with reference points close to , exhibit risk averse behavior and thus lower energy consumption. On the other hand, rational prosumers will start decreasing their consumption at cents, while risk seeking prosumers () will start decreasing their consumption at cents.
Fig. 6 shows the number of iterations needed for the best response algorithm to converge to a followers’ NE for different number of prosumers, under PT. Clearly, the best response algorithm converges, for all these cases. In addition, the number of iterations needed for convergence is reasonable, even as the number of prosumers significantly increases from to .
In this paper, we have proposed a novel framework for analyzing energy trading of prosumers with the power grid, while accounting for the uncertainty of the future price of energy. We have formulated the problem as a Stackelberg game between the power company (leader), seeking to maximize its profits by setting its optimal pricing strategy, and multiple prosumer (followers), attempting to choose the optimal amount of energy to trade. The prosumers game was shown to have a unique pure strategy Nash equilibrium under classical game theoretic analysis. Subsequently, we have used the novel concept of utility framing from prospect theory to model the subjective behavior of prosumers when faced with the uncertainty of future energy prices. Simulation results have highlighted the impact of behavioral considerations on the overall energy trading process.
As a future avenue of research, one can extend our model to a more dynamic multi-stage game that not only utilizes further capabilities of the storage devices (e.g. load shifting over time periods), but also admits efficient algorithms for obtaining its equilibrium points. In particular, devising incentive compatible mechanisms for our model in the form of multi-stage dynamic game is an interesting future problem.
-  S.-C. Chan, K. M. Tsui, H. Wu, Y. Hou, Y.-C. Wu, and F. F. Wu, “Load-price forecasting and managing demand response for smart grids: Methodologies and challenges,” IEEE Signal Processing Magazine, vol. 29, no. 5, pp. 68–85, 2012.
-  I. Atzeni, L. G. Ordóñez, G. Scutari, D. P. Palomar, and J. R. Fonollosa, “Demand-side management via distributed energy generation and storage optimization,” IEEE Transactions on Smart Grid, vol. 4, no. 2, pp. 866–876, June 2013.
-  E. Bitar, R. Rajagopal, P. Khargonekar, and K. Poolla, “The role of co-located storage for wind power producers in conventional electricity markets,” in Proceedings of the IEEE American Control Conference, San Francisco, CA, June 2011, pp. 3886–3891.
-  C. Wu, H. Mohsenian-Rad, and J. Huang, “Wind power integration via aggregator-consumer coordination: A game theoretic approach,” in Proc. IEEE PES Innovative Smart Grid Technologies (ISGT), Washington, DC, USA, January 2012, pp. 1–6.
-  Z. Chen, L. Wu, and Y. Fu, “Real-time price-based demand response management for residential appliances via stochastic optimization and robust optimization,” IEEE Transactions on Smart Grid, vol. 3, no. 4, pp. 1822–1831, 2012.
-  N. Hajj and M. Awad, “A game theory approach to demand side management in smart grids,” in Intelligent Systems’. Springer, 2014, pp. 807–819.
-  J. Chen, B. Yang, and X. Guan, “Optimal demand response scheduling with stackelberg game approach under load uncertainty for smart grid,” in In proceedings of the IEEE Third International Conference on Smart Grid Communications (SmartGridComm), 2012, pp. 546–551.
-  M. Yu and S. H. Hong, “Supply–demand balancing for power management in smart grid: A stackelberg game approach,” Applied Energy, vol. 164, pp. 702–710, 2016.
-  S. Maharjan, Q. Zhu, Y. Zhang, S. Gjessing, and T. Başar, “Dependable demand response management in the smart grid: A stackelberg game approach,” IEEE Transactions on Smart Grid, vol. 4, no. 1, pp. 120–132, 2013.
F.-L. Meng and X.-J. Zeng, “An optimal real-time pricing for demand-side management: A stackelberg game and genetic algorithm approach,” in
2014 International Joint Conference on Neural Networks (IJCNN), 2014, pp. 1703–1710.
-  W. Wei, F. Liu, and S. Mei, “Energy pricing and dispatch for smart grid retailers under demand response and market price uncertainty,” IEEE transactions on smart grid, vol. 6, no. 3, pp. 1364–1374, 2015.
-  M. Zugno, J. M. Morales, P. Pinson, and H. Madsen, “A bilevel model for electricity retailers’ participation in a demand response market environment,” Energy Economics, vol. 36, pp. 182–197, 2013.
-  N. Liu, X. Yu, C. Wang, and J. Wang, “Energy sharing management for microgrids with PV prosumers: A Stackelberg game approach,” IEEE Transactions on Industrial Informatics, 2017.
-  D. Kahneman and A. Tversky, “Prospect theory: An analysis of decision under risk,” Econometrica, vol. 47, no. 2, pp. 263–292, March 1979.
-  A. Tversky and D. Kahneman, “Advances in prospect theory: Cumulative representation of uncertainty,” Journal of Risk and uncertainty, vol. 5, no. 4, pp. 297–323, October 1992.
-  Y. Wang, W. Saad, N. B. Mandayam, and H. V. Poor, “Integrating energy storage into the smart grid: A prospect theoretic approach,” in Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, May 2014, pp. 7779–7783.
-  W. Saad, A. L. Glass, N. B. Mandayam, and H. V. Poor, “Toward a consumer-centric grid: A behavioral perspective,” In Proceedings of the IEEE, vol. 104, no. 4, pp. 865–882, 2016.
-  Y. Wang, W. Saad, N. B. Mandayam, and H. V. Poor, “Load shifting in the smart grid: To participate or not?” IEEE Transactions on Smart Grid, to appear, 2016.
-  Y. Wang, W. Saad, A. I. Sarwat, and C. S. Hong, “Reactive power compensation game under prospect-theoretic framing effects,” IEEE Transactions on Smart Grid, 2017.
-  L. Xiao, J. Liu, Y. Li, N. B. Mandayam, and H. V. Poor, “Prospect theoretic analysis of anti-jamming communications in cognitive radio networks,” in In proc. of the IEEE Global Communications Conference (GLOBECOM), 2014, pp. 746–751.
-  Y. Yang, L. T. Park, N. B. Mandayam, I. Seskar, A. L. Glass, and N. Sinha, “Prospect pricing in cognitive radio networks,” IEEE Transactions on Cognitive Communications and Networking, vol. 1, no. 1, pp. 56–70, 2015.
-  R. Yang, C. Kiekintveld, F. OrdóñEz, M. Tambe, and R. John, “Improving resource allocation strategies against human adversaries in security games: An extended study,” Artificial Intelligence, vol. 195, pp. 440–469, 2013.
-  D. Kar, F. Fang, F. Delle Fave, N. Sintov, and M. Tambe, “A game of thrones: when human behavior models compete in repeated stackelberg security games,” in Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 2015, pp. 1381–1390.
-  Q. Han, B. Dellaert, W. van Raaij, and H. Timmermans, “Integrating prospect theory and stackelberg games to model strategic dyad behavior of information providers and travelers: Theory and numerical simulations,” Transportation Research Record: Journal of the Transportation Research Board, no. 1926, pp. 181–188, 2005.
-  A.-H. Mohsenian-Rad, V. W. Wong, J. Jatskevich, and R. Schober, “Optimal and autonomous incentive-based energy consumption scheduling algorithm for smart grid,” in Innovative Smart Grid Technologies (ISGT), 2010. IEEE, 2010, pp. 1–6.
-  C. Wang and M. De Groot, “Managing end-user preferences in the smart grid,” in Proceedings of the 1st International Conference on Energy-Efficient Computing and Networking. ACM, 2010, pp. 105–114.
-  Z. Han, D. Niyato, W. Saad, T. Başar, and A. Hjorungnes, Game Theory in Wireless and Communication Networks: Theory, Models, and Applications. Cambridge University Press, 2012.
-  A. M. Fink et al., “Equilibrium in a stochastic -person game,” Journal of Science of the Hiroshima University, Series ai (Mathematics), vol. 28, no. 1, pp. 89–93, 1964.
-  L. S. Shapley, “Stochastic games,” Proceedings of the National Academy of Sciences, vol. 39, no. 10, pp. 1095–1100, 1953.
-  S. R. Etesami, W. Saad, N. Mandayam, and H. V. Poor, “Stochastic games for smart grid energy management with prospect prosumers,” arXiv preprint arXiv:1610.02067, 2016.
-  J. B. Krawczyk and S. Uryasev, “Relaxation algorithms to find Nash equilibria with economic applications,” Environmental Modeling and Assessment, vol. 5, no. 1, pp. 63–73, 2000.
-  F. Facchinei and C. Kanzow, “Generalized Nash equilibrium problems,” 4OR: A Quarterly Journal of Operations Research, vol. 5, no. 3, pp. 173–210, 2007.
-  J. B. Rosen, “Existence and uniqueness of equilibrium points for concave -person games,” Econometrica: Journal of the Econometric Society, pp. 520–534, 1965.
-  D. Bertsekas, A. Nedić, and A. Ozdaglar, Convex Analysis and Optimization. Athena Scientific, 2003.
-  R. T. Rockafellar and S. Uryasev, “Optimization of conditional value-at-risk,” Journal of risk, 2, pp. 21–42, 2000.
-  L. P. Metzger and M. O. Rieger, “Equilibria in games with prospect theory preferences,” 2010.
-  G. E. Rahi, A. Sanjab, W. Saad, N. B. Mandayam, and H. V. Poor, “Prospect theory for enhanced smart grid resilience using distributed energy storage,” arXiv preprint arXiv:1610.02103, 2016.
Appendix A Proof of Theorem 1
First, we analyze the strictly concave expected utility of prosumer in (8). By taking the second derivative of (8) with respect to , we get: , which is a strictly negative term, as . The optimal solution is either an interior point obtained by solving the necessary and sufficient optimality condition given by , or is at one of the boundaries, in case the interior solution is not feasible. Solving the optimality solution gives a unique solution . maximizes each prosumer’s expected utility function given that it lies in the feasible range of .
Appendix B Proof of Theorem 2
First, we show that the followers’ game is a concave game with closed and convex action sets in which the utility of player is a concave function of its own action , for any fixed actions of others . From (8), one can see that the utility function of each prosumer is quadratic, and thus concave, in terms of its own action variable . Moreover, the action set of each prosumer is clearly a closed convex set. Using [33, Theorem 1], we can show that the prosumers’ game admits at least one pure strategy NE. For NE uniqueness, we use [33, Theorem 2] to show that the prosumers game is diagonally strictly concave. This means that one can find a fixed nonnegative vector such that for every two action profiles , , where . We let for each . Using (8), we have
be the identity matrix, andbe a square matrix with all entries equal to 1. Then we can write , where . is a negative definite matrix due to the positive definiteness of and the fact that . By checking the diagonally strict concavity condition we get
where the last inequality is due to the negative-definiteness of the matrix . Using [33, Theorem 2] the NE will be unique.
Appendix C Auxililiary lemma for the proof of Theorem 3
There exists a constant for which the Nikaido-Isoda function associated with the prosumers’ game satisfies . Moreover, is convex in and strongly concave in such that
For any two action profiles of the prosumers and , the Nikaido-Isoda function adopted for the utility in (8) will be:
Using (30), for any two action profiles , we have
where the first inequality is due to the Cauchy-Schwarz inequality, and is an upper bound constant for the second term of the last equality. To show the convexity of with respect to , let be the matrix with all entries equal to 1. Using (30), a simple calculation shows that , where denotes the Hessian matrix of with respect to variable vector . Since and is a positive semi-definite matrix, this shows that , which implies is a convex function of . Finally using (8), one can easily check that the equality in (27) holds, which shows that is strongly concave with respect to its second argument . ∎
Appendix D Proof of Theorem 3
We show that , from Algorithm 1, where is a pure-strategy NE for the prosumers. To show that, we measure the distance of an action profile and its best response using the Nikaido-Isoda function and show that this distance decreases as becomes large. In particular, we show that at the limit, this distance equals zero which shows that the limit point is an NE of the game.
Using the first part of Lemma 1, we have
where the first inequality is due to the nonexpansive property of the projection operator, the second inequality uses the matrix norm inequality, and the last equality is obtained by replacing the expression for and noting that the induced 2-norm of matrix equals . Substituting (37) into (35) we have
Since , we can write
where the first equality is due to Lemma 1, and the last inequality is due to first part of Lemma 1 and the fact that maximizes . Multiplying both sides of the above inequality by and defining and , we get
Our goal is to show that for all , in which case by definition of we obtain . This not only shows that , implying that converges to a pure strategy NE of the prosumers game (note that if and only if is a NE), but it also shows that after steps, the action profile of the prosumers is an -NE of the game where (this is due to implies