The smart grid, a large-scale network of intelligent nodes that can communicate, operate, and interact autonomously for reliable and efficient power delivery, is envisioned to be a secure and self-healing power network for the 21st century, incorporating various sources of energy . In the smart grid, with the two-way communication infrastructure, consumers are increasingly becoming more proactive. Demand-side management (DSM) is an essential part of this transition. Under the umbrella of DSM, there are both technical and social programs. The common aim of these programs is to help improve energy efficiency in both the short term and the long term. An overview of DSM in smart grids can be found in . An early tutorial on the demand-side view of electricity markets can be found in .
One aspect of DSM is demand response, which is defined as the response of consumers’ demands to price signals from the utility companies . Demand response allows companies to manage the consumers’ demands, either directly (through direct load control) or indirectly (through pricing mechanisms). Demand response comes with great benefits. For example, it has been shown that demand response programs improve the electricity market efficiency 
. Furthermore, the Federal Utility Regulatory Commission estimates that demand response programs will reduce the peak load by 4-9% in the United States by 2019. A comprehensive survey on the pricing methods and optimization algorithms for demand response programs can be found in . For an overview of the methodologies and the challenges of load/price forecasting and managing demand response in the smart grid, see .
With the internationalization of energy markets and the deep penetration of renewable and distributed energy resources, consumers are increasingly having more options in terms of where to buy their energies, and some of them are becoming prosumers. This makes investigating load adaptive pricing mechanisms in energy systems important. Using the framework of game theory, load adaptive pricing has been introduced decades ago. In this paper, we utilize tools from game theory to design a multi-period demand response management program at which multiple companies (energy sellers) and consumers (energy buyers) interact and reach an equilibrium point at which prices and demands are optimally chosen. While our mathematical analysis is general and applicable to various smart grid setups, for the purpose of this paper, one can think of “company” as a utility company serving households, businesses, and industrial consumers.
While many energy consumers around the world have access to only one company, alternative structures are now becoming a reality . For example, a company called LO3 Energy has begun setting up a small-scale grid operated by consumers that allows peer-to-peer transactions between distributed energy resource owners and demanders in the neighborhood . The emergence of such alternative structures motivate us not to limit our contribution to the classical single-company-multi-consumer scenario. Furthermore, in a smart grid where consumers can simultaneously change their sources of energy, competition between the owners of these energy sources arise, leading to at least partially conflicting objectives between various energy owners, which makes applying tools game theory natural. With the use of game theory, advances in local energy trading considering such possible conflicts are made . For a comprehensive survey of game-theoretic methods for the smart grid, we refer to .
In the smart grid, temporal variations play a critical role on both the supply side and the demand side. On the supply side, it can be more costly to produce one unit of power in a hot summer afternoon than later in the same day. Furthermore, temporal variations also affect the available power from renewable sources. On the demand side, consumers typically use more energy during the day than in the evening. Such variations also make demand response programs important, as they provide economic incentives to consumers to shift some of their consumption. Accordingly, in this paper, we let our game-theoretic approach to also incorporate different time periods. Also, we investigate both analytically and numerically, how the number of periods considered in the game affect the outcomes at equilibrium.
In a nutshell, this paper introduces a game-theoretic approach for multi-period demand response management to capture the interactions among multiple companies (energy sellers who choose their prices to maximize their revenues, in addition to allocating their power) and their consumers (energy buyers who optimally respond to price signals, given their minimum energy needs across the time horizon). We derive all optimal (equilibrium) decisions in closed form, and provide a distributed algorithm to guarantee the preservation of privacy. Additionally, we use real demand response data to demonstrate the applicability of our approach. Finally, we also study the asymptotic behavior as the number of consumers and the number of periods grows.
State of the Art. The use of game theory in the smart grid has attracted significant attention in the literature [13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38]. Application examples include: the control of small scale power systems , management of energy exchange between microgrids , and minimizing communicational delay between smart elements , and many others .
In this paper, we focus on DSM and demand response applications [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38]. These applications have demonstrated that game theory can improve the reliability and efficiency of the grid, while maintaining economic incentives. However, to the best of our knowledge, the vast majority of these contributions are either limited to a single seller case, or a single period one. Furthermore, they primarily focus on either the supply-side or the demand-side. Our goal here is to provide insights on the multi-seller-multi-period case while capturing the interactions between the supply and demand sides.
There are several papers in the literature that have addressed inter-temporal considerations in DSM and demand response [17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. An autonomous DSM through scheduling of appliances has been implemented within a noncooperative game framework in . The participants in the game are energy consumers who are connected to the same utility company, and the outcome of the game is the power consumption schedule of appliances that minimizes the overall energy cost. A more recent extension adds energy storage into the picture , where a Stackelberg game was developed between the utility company and the end-consumers. Demand response scheduling with multiclass appliances with different levels of elasticity has been recently studied in . In [20, 21], noncooperative games to reduce peak-to-average ratio have been proposed. The authors in  showed that it is also possible to reduce peak-to-average ratio via a repeated game framework. A four-stage Stackelberg game has been studied where three stages are at the leader level (the utility retailer), and the fourth stage is at the consumer level . The retailer chooses the amount of energy to procure, and the sources to produce it, in addition to deciding on the price. Consumers respond to these prices through demand selection. The authors in  studied a user-centric differential game at which consumers allocate their powers across their household devices. We note that demand response can in fact be affected by wholesale electricity markets. For example, in , day ahead dynamic pricing for demand response has been introduced, where issues related to wholesale price fluctuations faced by the retailer are studied. Also, the authors in  have let a utility retailer to act as intermediary player between end consumers (who respond to price signals) and the wholesale market. While the contributions in [17, 18, 19, 20, 21, 22, 23, 24, 25, 26] are important and reveal the importance of game theory for multi-period considerations in demand-side management, they are all limited to a single seller/utility/retailer case.
A considerable number of contributions have used game theory to analyze cases where there are multiple sellers/utilities/retailers [27, 28, 29, 30, 31, 32, 33, 34]. For example, analysis of how plug-in hybrid electric vehicles can sell back to the grid has been explored in [27, 28, 29]. Similar analysis has also been carried out for electric bicycles . A two-level game (a noncooperative game between multiple utility companies and an evolutionary game for the consumers at the lower level) has been proposed in . The authors in  instroduce a Stackelberg to capture the interactions between electricity generator owners and a demand response aggregator. In , a distributed game between energy consumers of different types (sellers who have surplus of energy, and buyers who have energy need) has been designed while emphasizing individual preferences. Furthermore, in , the analysis of three-party energy management scheme between residential users, a shared facility controller, and the main power grid, has been conducted via a Stackelberg game. These works [27, 28, 29, 30, 31, 32, 33, 34] have demonstrated the usefulness and the strength of game theory in capturing the interplay between buyers and sellers in the smart grid, but they are mainly focused on single period setups.
Among the contributions in the literature the ones most relevant to this paper are [35, 36]. A Stackelberg game for demand response management with multiple utility companies has been proposed in , where consumers choose their optimal demands in response to prices announced by different utility companies. This Stackelberg game was shown to have a unique Stackelberg equilibrium at which utility companies maximize their revenues and end-consumers maximize their payoff functions. In this framework, utility companies were the leaders of the game and consumers were the followers. In , an extension to the large population regime was carried out. In this game, the utility companies aim to maximize their profits, while the end-consumers wish to maximize their welfare. It was shown in that paper that a unique number of utility companies exists for which profits are maximized. A variation of  to a user-centric approaches are discussed in [38, 37]. These works [35, 36, 38, 37], even though effectively capturing consumer-utility interactions, are limited to the single-period scenario.
Motivated by the limitations of existing works to single periods/companies, this paper proposes an analytical multi-period-multi-company game-theoretic framework for demand response management in the smart grid. Such a generalization enables us to analytically study the effects of market competition between companies, along with the multi-period considerations at both the demand-side and the supply-side and the interactions between them.
Contributions. Our multi-period model and contributions differ from those in the single-period works in [35, 36] at the consumer-side, company-side, and the overall interaction. At the consumer-side, we have an additional minimum energy constraint that needs to be satisfied across all periods, and also achieve billing minimization. We also prove, bot theoretically and numerically, that our generalization provides desirable incentives (financial and in terms of satisfaction) for consumers. At the company-side, we study an optimal power allocation game over the time-horizon, and reveal desirable properties that allow for the accommodation of company-specific operational needs and profit-maximization. For the overall interaction, we provide an alternative computationally cheap closed-form solution for the prices, utilize real data to demonstrate the fast convergence to optimal decisions while preserving privacy, and study the asymptotic behavior as the number of periods or consumers grows. Our analysis is general and makes it possible to also accommodate various future inter-temporal considerations.
We formulate in this paper a Stackelberg game for multi-period-multi-company demand response management. We derive solutions in closed form and find precise expressions for maximizing demands at the consumers’ level, and the revenue-maximizing prices for the companies. We also prove the existence and uniqueness of the Stackelberg equilibrium, and propose a distributed algorithm to compute it using only local information. Furthermore, we exploit the closed-form expressions to formulate a new power allocation game, we prove the existence and uniqueness of a pure-strategy Nash equilibrium of the power allocation game, and find its analytical expression. In the large population regime, we find an optimal company-to-consumer ratio. Furthermore, we demonstrate the applicability of our game to real life data. Our work captures the competition between companies, budget limitations at the consumer-level, energy need for the entire time-horizon.
Some of the material in this paper was presented earlier in the conference paper , but this paper provides a more comprehensive treatment of the work. The major improvements and novelties are as follows:
Power Allocation: We formulate and solve a power allocation game at the utility companies’ level. This game addresses the following question: How can each utility company optimally allocates its power availability over the entire time horizon?
Asymptotic Behavior: We study the asymptotic behavior as the number of periods grows and prove that this provides more incentives for consumer participation in demand response. We also study the large population regime and find the optimal company-to-consumer ratio.
Privacy Preservation: In , the closed-form solutions require each company to know the power availability of other companies. We resolve this issue here by providing a distributed algorithm that converges to optimal prices.
Case Studies Using Real Data: In , simulations showed how multi-period demand response provides incentives for consumers’ participation. In this work, we show that this also holds using real life data. And we further study the behavior of the prices, the effect of varying the budgets, savings for consumers, and demonstrate the fast convergence of our distributed algorithm.
Proofs: Our conference paper only stated the results up to Theorem 2, without proofs. This manuscript includes a much more comprehensive treatment, containing all proofs and derivations of the results.
Organization. The remainder of the paper is organized as follows. Prelimenaries from game theory are provided in Section II. The problem is formulated in Section III, and optimal prices and demands are analyzed via a Stackelberg game in Section IV. In Section V, a power allocation game at the companies side is formulated based on the closed-form solutions of the Stackelberg game. Next, we provide a distributed algorithm for the computation of all optimal strategies using local information in Section VI. The asymptotic behavior is studied as the number of periods or the number of consumers grows in Section VII. Next, we present results on case studies using real demand response data in Section VIII. Finally, we conclude the paper in Section IX with a recap of main points and identification of future directions. An appendix at the end provides details of proofs of the five theorems and some auxiliary results.
Ii Preliminaries from Game Theory
A static -person noncooperative game is comprised players set, actions sets, and utility functions. Let the players set be denoted by , where is the number of players. Each player has an action set , and the decision of player is denoted by
. The vector of decisions taken by other players is. Each player aims to maximize his/her utility function . An equilibrium concept that is suitable for such games is the Nash Equilibrium (NE), which is defined below.
Definition 1: The action vector constitutes a Nash equilibrium for the -person static noncooperative game in pure-strategies if
Sometimes it would be beneficial to allow for hierarchy in the decision process. In such a case, there are two types of players, leaders and followers. The leaders’ decisions are more dominant, and the followers respond to the decisions taken by the leaders. This kind of hierarchal games is called Stackelberg games, and the corresponding solution concept is called the Stackelberg equilibrium. The leaders have the privilege of choosing how to take their actions at the beginning of the game. However, they have to take into account how the followers would respond to these actions and how each leader’s decision is influenced by the decisions of the other leaders. To be more precise, suppose that we have leaders and followers. Denote the followers set by , and the leaders set by , with action sets and , respectively. Denote the action of leader by , and the action of follower by . The vector of actions taken by all leaders is . The utility of leader is denoted by , where denotes the decisions of the other leaders, and .
Definition 2: The action vector is a Stackelberg Equilibrium strategy for all the leaders in pure-strategies if, for each ,
where is the optimal response by all followers to the leaders’ decisions (under the adopted equilibrium solution concept at the followers level). For a Stackelberg game, the pair () constitutes the equilibrium strategy.
Iii Formulation of a Mathematical Model
Let be the set of companies, be the set of consumers, and be the finite set of time slots.
We formulate a static Stackelberg game between utility companies (the leaders) and their consumers (the followers) to find revenue maximizing prices and optimal demands. In Stackelberg games, the leader(s) first announce their decisions to the follower(s), and then the followers respond. In our game, the leaders send price signals to the consumers, who respond optimally by choosing their demands. To capture the market competition between the utility companies, we let them play a price-selection Nash game. The equilibrium point of the price-selection game is what utility companies announce to their consumers. Figure 1 illustrates the hierarchical interaction between companies and consumers.
In the parlance of dynamic game theory , we are dealing here with open-loop information structures, with the corresponding equilibrium at the utilities level being open-loop Nash equilibrium. Therefore, this is a one-shot game at which all the prices for all the periods are announced at the beginning of the game, and the followers respond to these prices by solving their local optimization problems.
Consumer-Side. Because of energy scheduling and storage, consumers may have some flexibility on when to receive a certain amount of energy. We are concerned about the total amount of shiftable energy. For non-shiftable energy, one can add some period-specific constraints. Each energy consumer receives all price signals from each company at each time slot and aims to select his corresponding utility-maximizing demand for each time slot from each company, subject to budget and energy need constraints. Denote company ’s price at time by . Let and denote, respectively, consumer ’s budget and minimum energy need for the entire time-horizon. The utility of consumer is defined as
where and are preference parameters. Note that if or , the utility of the consumer becomes negative, which is not realistic for demand response applications, and hence we take and . A typical value for is , but we still solve the problem for any to keep it general. The logarithmic function (3) is known to provide proportional fairness and is widely used to model consumer behavior in economics [41, 30]. It has been validated to provide good demand response [30, 42, 37, 43, 44, 45, 35]. Our analysis in this paper is quite general and can be used in any market arrangement with multiple sellers and buyers under budget limitations and capacity constraints.
Consumer aims to achieve the highest payoff while meeting the threshold of minimum amount of energy and not exceeding a certain budget. To be more precise, given , , and , the consumer-side optimization problem is formulated as follows:
Note that there is no game played among the consumers. Each consumer responds to the price signals using only her local information. We indirectly handle consumers’ cost minimization via our analysis in later sections.
Company-Side. Letting denote the prices set by other companies, the total revenue for company is given by
Given the power availability of company at period , denoted by , and for a fixed , company ’s problem is;
The goal of each company is to maximize its revenue and hence maximize its profit. Additionally, because of the market competition, the prices announced by other companies also affect the determination of the price at company . Thus, company ’s price selection is actually a response to what other competitors in the market have announced; this response is also constrained by the availability of power. Thus, what we have is a Nash game among the companies. We emphasize that while company ’s problem is affected by what its competitors decide, we can still achieve the equilibrium strategies using only local information, via our distributed algorithm discussed later in Section VI. Finally, while at this point we have ’s fixed, we will later formulate a power allocation game to optimally choose them.
Iv Demand Selection and Revenue Maximization (Stackelberg Game)
In this section, we solve the above optimization problems in closed form and demonstrate the solutions are unique.
Consumer-Side Analysis. We start by relaxing the minimum energy constraint (6). For each consumer , the associated Lagrange function is given as follows:
where ’s are the Lagrange multipliers. The KKT conditions of optimality in this case are sufficient because the objective function is strictly concave and the constraints are linear , and solving for them leads to
The following theorem, whose proof can be found in the Appendix, states the necessary and sufficient condition for so that the above ’s meet the minimum energy constraint (6).
The above theorem can be interpreted as billing costs minimization. The equality in (11) corresponds to the case at which consumer minimizes his billing cost subject to the energy need constraint, and it can be also computed locally. We later demonstrate that using (11) leads to savings that can exceed
Company-Side Analysis We apply the demands derived in the consumers-side analysis (which were functions of the prices) and show that optimality is achieved at the equality of constraint (9). We start by solving for prices that satisfy the equality at (9) and then prove that they are revenue-maximizing, strictly positive, and unique. Consider the equality in (9), and by the optimal demands (10), there holds
for all . Let and . Then, for each company ,
The equations in (12) can be combined into a linear equation , where is a matrix whose diagonal entries are , , , and off-diagonal entries all equal to , is a vector in stacking , , , and a vector in whose entries all equal to .
We have the following results (proofs are in the Appendix).
The matrix is invertible.
The prices that solve (12) are strictly positive and uniquely given by
In practice, due to production costs and market regulations, cannot be outside the range of some lower and upper bounds for all and , as in . If , then is set to , and similarly for the upper-bound, if , then we set .
Letting for each consumer, the value of coincides with . In this case, by (13), we observe that for any given ’s,
is a constant for all and . Thus, the power availability is inversely proportional to the prices.
Lemma 2 provides a computationally cheap expression for the prices. Since can be directly computed using (13), there is no need to numerically compute or . This enables us to deal with a large number of periods or utility companies, without worrying about computational complexity.
Existence and Uniqueness of the Stackelberg Equilibrium. Denote the strategy space of utility company (a leader in the game) at by . The strategy space of for the entire time horizon is . and the strategy space of all companies is .
For given price selections , the optimal response from all consumers is
where for each , is the unique maximizer for and is given by (10). This now leads to the following theorem, whose proof can be found in the Appendix.
At the Stackelberg equilibrium, it can be easily verified that
One observation is that when a company gains in terms of revenue, the same amount must be lost by other companies because the sum of revenues is a constant, which demonstrates a conflict of objectives between utility companies. However, by the definition of the equilibrium strategy, this is the best each company can do, for fixed power availabilities ’s. But, given a total amount of available power, , a company has across the time horizon, it is possible that it gains in terms of revenue by an efficient power allocation. This motivates us to formulate a power allocation game and analytically answer the following question: How can company allocate its power so that it maximizes its revenue?
V Power Allocation (Nash Game)
In this section, we exploit the closed-form solutions for consumer demands and companies’ prices to formulate and solve a power allocation game for companies. For the remaining part of this paper, and for the purpose of simplifying the analysis without losing the main insights, we assume that for each consumer , we have .
Given the power availability from other companies, , and since the equality in (9) is satisfied at equilibrium, the revenue function of company can be represented as
The optimal prices (13) are functions of ’s, leading to
Denote the action set of company at time by . Since ’s are non-negative, we have , for any and any . Thus, given , the optimization problem for company is as follows:
The above problem is only applicable for the case at which generation is fully controllable, for example, it can be from dispatchable generators. For the smart grid, because of the various generation sources, full-controllability does not always hold, and in fact, for renewable resources it could be completely gone. We demonstrate the ability to relax this assumption later.
Existence and Uniqueness of Nash Equilibrium. The following theorem, whose proof can be found in the Appendix, states the existence and uniqueness of Nash equilibrium, and provides an expression for it.
There exists a unique pure-strategy Nash equilibrium for the power allocation game, and it is given by
The proof of Theorem 3 reveals that (16) is strictly concave and increasing in each . This is an important property that allows accommodating further company-specific operational constraints and relaxing the full-controllability assumption. To illustrate, suppose that company has a mix of generation sources for which generation is controllable for some periods and only partially controllable for others. Then, it can add period-specific bounds. Existence and uniqueness of a pure-strategy Nash equilibrium are still guaranteed due to the strict concavity . Since generation costs are typically assumed to be convex, company can also allocate its generation to maximize its profit, by subtracting the cost from (16), and if the cost is strictly convex, again there exists a unique pure-strategy Nash equilibrium.
Vi Privacy-Preservation (Distributed Algorithm)
The NE for the power allocation game given by (18) can easily be computed by each company using its local information. Moreover, for energy users, it can be seen from (10) that in the computation of user ’s optimal demand selection, no information from other users is needed, and user ’s local information would suffice for optimal response. However, the closed-form solution for optimal prices ’s given by (13) requires each company to know consumers’ budgets and the power availability of all the other companies. Companies might not want to share such information with each other. To avoid such a privacy concern, we propose a distributed algorithm that allows companies to compute their optimal prices using only local information, and show that this algorithm converges to the optimal prices given by (13). This algorithm, combined with utility-maximizing demands given by (10) and the NE given by (18), leads to the computation of all the optimal strategies with only local information at both the company level and the consumer level.
For each iteration , denote the demand from user at time from company by , and the price announced by company and time by . In our algorithm, is chosen arbitrarily for each company and time . Based on the initial price selection, is computed using (10). Then, the prices are sequentially updated using the following update rule:
where is appropriately selected for company at time in iteration , and we find an expression for it as a function of in Theorem 4. Whenever a company updates its price at time , it transmits the price to each consumer , and they modify their demands accordingly. Once prices converge to their optimal values, users optimally respond by (10) and the algorithm terminates. We have the following theorem for the convergence of the algorithm; its proof can be found in the Appendix.
For each company at time in iteration , if the prices are sequentially updated using (19) such that
where , then Algorithm 1 converges to optimal prices.
Vii Asymptotic Regimes
In this section, we study the asymptotic behavior of the equilibrium strategies for the demands, prices, and power allocation, given by (13), (10), and (18), respectively. Particularly, we study how the payoffs, revenues, prices, and demands are affected as . Moreover, we find an appropriate company-to-user ratio for the large population regime.
When the Number of Periods Grows. Suppose all companies have the same total power availability . In this case, we have
and the payoff of user becomes
in which is positive. Thus, as increases, the multiplicative term of the logarithmic function increases at a faster rate than the decrease of . Hence, as increases, the equilibrium utility of each user monotonically increases. Taking the limit, it can be verified that
Furthermore, note that the demand from user from company at time converges to zero as . We claim that the revenues are constants. To see this, recall that
which is a constant since both the number of companies and the budgets of the users are fixed.
Note that the limit point of the utility function of user is the proportion of his budget to the total budgets times the total power availability. So if a particular user has of the sum of all the budgets, he gets of the available power. Furthermore, the revenue for each company is the proportion of the sum of the budgets to the number of companies. In addition, the demand by user from company at time is the proportion of his budget to the total budgets times the total power availability at from .
When the Number of Consumers Grows. When the number of consumers increases, each additional user has some budget , and since the total power availability is fixed, competition among users arises on the same amount of power and hence utility companies will increase their revenue-maximizing prices.
We start by assuming that the budget for each user is the same, and then increase the number of users and see what happens as . We also keep the assumption that is the same for all companies. In this case, the optimal prices and demands become
Clearly, as and as . When the population is large and the power availability is fixed, it is not surprising that because the portion each user can get from the available power gets smaller and smaller as increases. Furthermore, it can be easily verified that and . Thus, with the limit points resulting in unrealistic outcomes, a balance between the supply and demand needs to be achieved, which we achieve by finding an appropriate company-to-user ratio.
Now, the question we ask is: for a given maximum allowable price , call it , what is the appropriate company-to-user ratio ? If there are more companies than necessary in the market, there will be losses in terms of revenues. On the other hand, if there are fewer companies than necessary, the prices can exceed , leading to undesirable outcomes. The following theorem, whose proof can be found in the Appendix, provides an optimal ratio at which prices do not exceed and the revenues being maximized while satisfying the equality in (14).
Suppose that the total power availabilities for all companies are the same. Then, at the NE of the power allocation game, and at the Stackelberg equilibrium of the price and demand selection game, the optimal prices ’s given by (13) satisfy
if, and only if,
Viii Case Studies
In this section, we present results on some case studies on representative days from a Dutch smart grid pilot  and the EcoGrid EU project . We numerically study optimal prices and demands, and their corresponding payments and utility functions. We also show how our approach results in monetary savings for consumers. Furthermore, we show that increasing the number of periods provides more incentives for consumers’ participation in demand response management. Additionally, we demonstrate the fast convergence of our distributed algorithm to optimal prices. We also release an open-source interactive tool containing the simulations using Python and Jupyter notebooks  in .
EcoGrid EU Project. This demand response project was conducted from March 2011 to August 2015 in Bornholm, Denmark. The number of consumers in this experiment was approximately . For a representative day (December 5th, 2014), we apply our method to hourly prices and shiftable demand consumption from this experiment111Due to the unavailability of data in raw format, in this section, we take estimates of the data points from publicly available figures.. The experimental prices are in and we scale them to . We start by assuming that there is only one company () and letting the consumers to be homogeneous (they have the same budgets and energy need) with , and then generalize the results to and heterogeneous consumers. Since we are taking hourly prices for a day, we have .
Finding the necessary parameters
In our model, for each period , we have a fixed power availability on the supply-side. Also, for each consumer , his minimum demand and budget are fixed for the entire horizon. These are necessary parameters that need to be known to solve for optimal demands and prices. We let the power availabilities ’s match the experimental hourly variation of the total load (this is consistent with our model, as we primarily focus on shiftable consumption). For the entire time-horizon, we have Since consumers are homogenous, we have Next, using Theorem 1, we plug-in and the experimental hourly prices in (11) to find the minimum budget need, which is .
In Figure 2, we plot the total power availabilities
’s, the prices found experimentally and using the Stackelberg game, and the corresponding total payments by all consumers for their demands. Our approach leads to prices that have a slightly smaller mean as in the experiment and a significantly smaller variance. One advantage our approach has is that it results in billing savings for consumers, as we show in Figure2 (this demonstrates the importance of Theorem 1, which we use to find the minimum budget need for the consumers). Note that the net demands for each time period are the same for the Stackelberg game as for the experimental ones (so, consumers receive the same amount of energy for smaller costs). This would lead to more monetary incentives for active consumer participation in demand response management, while being consistent with the company’s objectives, since the Stackelberg game prices found using (13) are revenue-maximizing as shown in the proof of Theorem 2.
Next, we make consumers heterogeneous and increase the number of companies. We differentiate between consumers by varying their budgets, and take 5 classes of consumers, as in the EcoGrid EU experiment. We let consumers’ budgets be , , , , and . We also let the number of companies be , which is consistent with the actual energy sources used in the experiment. Precisely, the system is powered by 61% wind energy (), 27% biomass (), 9% solar energy (), and 3% biogas (). We split the total power () among the energy sources according to experimental proportions, assuming that each energy source is owned by a single company that acts as a company in our game.
With the above setup, we study the effect of varying the number of periods from to . To do this, we need to find a way for companies to allocate their total power across the time horizon for each fixed , which can be done by using Theorem 3, which states that equally splitting the total power across the time horizon for each company constitutes a unique Nash equilibrium for the power allocation game (it is also shown to be the global maximizer in the proof).
Figure 3 shows the influence of varying the number of periods on prices, power allocated, revenues, and consumer utilities. We observe that as increases, the power allocated at each period gets smaller and smaller. On the other hand, prices can increase or decrease, dependent on the company, and they converge to positive constants. Furthermore, revenues might also increase or decrease, dependent on the company (note that the company that achieves the highest revenue is the one that offers the lowest prices, and vice-versa). In view of (14), the sum of revenues at equilibrium is a constant that matches the sum of all consumer budgets. And hence, whenever the revenue increases (decreases) for a company , at least one other company will incur a loss (gain) in terms of revenue. None of the companies can do better by altering its power availabilities across the time horizon, nor by changing its prices. This follows from the definition of Nash equilibrium. Furthermore, we note that the revenues are proportional with the total capacity, and the company with highest (lowest) portion of the market is the one that incurs the largest increase (decrease) in revenue. Changes seem to saturate beyond .
Interestingly, in Figure 3 we observe that as increases, the utilities for consumers also increase, and hence they will be more attracted to demand response programs, which is desirable . In comparison with the single-period setup [35, 36], this shows that the multi-period demand response provides improvements on the consumers’ end. This increase, however, does not change significantly after a certain number of periods, and we note that the choice of seems to be appropriate and meaningful as it can represent hourly pricing for a day.
To demonstrate the performance of our algorithm, we take the case when and study the algorithm’s performance for different values of in Figure 4. When , we observe that the algorithm converges very fast to the optimal prices and takes about less than iterations to reach equilibrium. The values are consistent with the values in Figure 3 when , where we used the analytical expressions of the prices. Next, we increase to and observe that the algorithm converges at a lower rate, but still very fast. Thus, the rate of convergence is inversely proportional to the value of . However, when decreases to a negative value, there are no guarantees on convergence. For example, if we take , the algorithm diverges. Theorem 4 only guarantees the convergence of the algorithm when . We have verified that our distributed algorithm converges very fast for various values of and alternative values of and , and the reader might experiment with varying them using our open-source code in .
Dutch Smart Grid Pilot. This experiment was conducted in Zwolle, the Netherlands, for about one year (May, 2014 to May, 2015). Dynamic tariffs and smart appliances were used to study the responsiveness of residential electricity demand. Tariffs were announced to consumers a day ahead. For a group of consumers with modal incomes, we study the behavior of the average consumer’s demand and payments using experimental prices and the prices derived using our method. Here, we take , which is consistent with the Dutch pilot. Also, the experimental prices are in . Furthermore, we take the consumers to be homogenous.
Finding the necessary parameters
We find the parameters as in the EcoGrid EU experiment with some slight modifications. We approximate the net experimental demands by consumers for each period , and assume that on the supply-side matches our approximation. For each consumer , we let be equal to the average consumer’s net demand for the entire day (here, ). Then, we plug-in and the experimental hourly prices in (11) to find the minimum necessary daily budget, which is .
Using the above parameters, we again use (10) and (13) to find optimal demands and prices. In Figure 5, we plot the average consumer’s hourly demand, the prices found experimentally and using the Stackelberg game, and the corresponding total payments by the average consumer. We again observe that our approach leads to smaller prices with a significantly smaller variance. For the average consumer, we observe that significant savings can be achieved using our approach (more than ). Next, we study the performance of our distributed algorithm in Figure 6. As in the case of the EcoGrid EU experimental data, our algorithm achieves fast convergence to optimal prices using only local information.
Ix Conclusion and Research Directions
In this paper, a privacy-preserving multi-period demand response problem has been studied. We have captured the interactions between companies and energy consumers, and found optimal prices and demands. Based on the closed-form expressions, a power allocation game for companies has been formulated and solved. Furthermore, a distributed algorithm has been proposed to compute all equilibrium strategies using only local information. Moreover, the asymptotic behaviors as the number of periods increases. In the large population regime, an appropriate company-to-user ratio has been derived to maximize the revenue of each individual utility company. The paper has shown that the multi-period scheme provides more incentives for the participation of energy consumers in demand response management, which is of critical importance , and we have also conducted case studies using real data.
For future work, it would be interesting to include energy scheduling and storage and study their influence on optimal demands and prices, in addition to including company-specific operational constraints. The game studied in this paper is multi-period but essentially static. Therefore, using tools from dynamic game theory  is another possible direction.
Proof of Theorem 1. Note that
is the same as
By (10), this is equivalent to
Proof of Lemma 1. The matrix can be represented as
Note that is invertible. Furthermore,
Since and , each element in the summation is less than and overall value of the summation is less than , and this clearly leads to . By Sherman-Morrison Formula , if , then
Thus, is invertible and we can apply (26).
Proof of Lemma 2. By Lemma 1, the prices are uniquely given by , and by using (26), the price selection for each at is
Strict positivity follows from
Proof of Theorem 2.
which is concave (linear) in each . Thus, by the compactness of , there exists a pure-strategy Nash Equilibrium (NE) . Next, suppose that a company deviates from (13) and announces a price of at a fixed time . If , then
where the inequality follows from . Thus, has no incentive to increase the prices from (13). Furthermore, since the prices given by (13) are attained the equality of the capacity constraint in (9), company has no incentive to choose because it will not result in selling more energy. Therefore, for every period , company does not benefit from deviating from (13). Hence, the prices given by (13) maximize the revenues and constitute a NE.
Proof of Theorem 3. Note that (16) is equivalent to
where Note that and it depends on the strategies of other companies and it is fixed for company . A pure-strategy Nash equilibrium exists if is concave in each for each company and if is a compact subset of . Since it is clear that is compact, it is enough to show concavity of . From (27), via a sequence of simple tricks,
Note that and