Dissecting demand response mechanisms: the role of consumption forecasts and personalized offers

by   Alberto Benegiamo, et al.

Demand-Response (DR) programs, whereby users of an electricity network are encouraged by economic incentives to rearrange their consumption in order to reduce production costs, are envisioned to be a key feature of the smart grid paradigm. Several recent works proposed DR mechanisms and used analytical models to derive optimal incentives. Most of these works, however, rely on a macroscopic description of the population that does not model individual choices of users. In this paper, we conduct a detailed analysis of those models and we argue that the macroscopic descriptions hide important assumptions that can jeopardize the mechanisms' implementation (such as the ability to make personalized offers and to perfectly estimate the demand that is moved from a timeslot to another). Then, we start from a microscopic description that explicitly models each user's decision. We introduce four DR mechanisms with various assumptions on the provider's capabilities. Contrarily to previous studies, we find that the optimization problems that result from our mechanisms are complex and can be solved numerically only through a heuristic. We present numerical simulations that compare the different mechanisms and their sensitivity to forecast errors. At a high level, our results show that the performance of DR mechanisms under reasonable assumptions on the provider's capabilities are significantly lower than



There are no comments yet.


page 1

page 2

page 3

page 4


A Truthful FPTAS Mechanism for Emergency Demand Response in Colocation Data Centers

Demand response (DR) is not only a crucial solution to the demand side m...

Demand Response For Residential Uses: A Data Analytics Approach

In the Smart Grid environment, the advent of intelligent measuring devic...

Learning Dynamical Demand Response Model in Real-Time Pricing Program

Price responsiveness is a major feature of end use customers (EUCs) that...

Limiting gaming opportunities on incentive-based demand response programs

Demand Response (DR) is a program designed to match supply and demand by...

Incentive Compatibility, Scalability and Privacy in real time Demand Response

The high penetration of Renewable Energy Sources in modern smart grids n...

Semi-Distributed Demand Response Solutions for Smart Homes

The Internet of Things (IoT) paradigm brings an opportunity for advanced...

Functional Model of Residential Consumption Elasticity under Dynamic Tariffs

One of the major barriers for the retailers is to understand the consump...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Demand Response (DR hereinafter) programs are envisioned to be a key feature of the Smart Grid paradigm [1]. By means of economic incentives (discounts or penalties), DR schemes encourage users to rearrange their consumption in response to the network state, thus mitigating the grid overload and driving wholesale prices down.

Several analytical models are available in the literature, which describe and quantify the effects of DR mechanisms. Whatever their specifics are, these schemes need to model how users react to the incentives. Ideally the models should capture the most realistic features of a practical DR mechanism while maintaining tractability.

Among these contributions, the authors of [2] study how an energy provider should select time-dependent discounts to minimize its production costs. They assume that the percentage of users who shift their consumption from slot to slot is a decreasing function of the temporal distance between slots and and a concave and increasing function of the discount offered in slot (), independent from discounts in other slots. The same user’s model as in [2] is adopted also in [3], where the optimization problem is extended in order to account for battery storages and distributed renewable sources available into a specific microgrid. Authors of [4] propose a day ahead pricing scheme which maximizes the provider’s profitability and capacity utilization. Users are assumed to reschedule their consumption by comparing the utility they get by scheduling a task in each timeslot ; therefore they allocate their consumption proportionally to these utilities, i.e., they consume a fraction of their total energy demand in timeslot . The resulting optimization problem is non convex but some relaxation techniques are introduced, which allow one to calculate a solution within a reasonable amount of time. In [5], a more realistic model is proposed where each user first calculates the welfare (defined as utility minus time-dependent cost) she gets from consuming electricity in each of the possible timeslots, and then allocates all the consumption to the slot returning the largest welfare. As we show below (see Sec. 4.4) this model can lead to a much more complex optimization problem than the one presented in [5]. Finally, the authors of [6] propose a full-fledged game theoretical model, but their results hold only if users experience a large number of interactions without any change in the system.

We claim that these studies rely on too strong assumptions, which jeopardize their usability for practical purposes. Interestingly, we observe that the assumptions are sometimes hidden in the macroscopic models the papers start from. In particular in this paper we focus on [2] and show that its model requires personalized offers and a very precise forecast of the baseline consumption of each user. The implementation of these features may require potentially significant costs in terms of communication, measurement and computation infrastructure. Besides highlighting these implicit requirements in the analytical framework in [2] (and then also in [3]), we explore their potentials considering four DR mechanisms with different levels of complexity:

  1. the base mechanism corresponds to an optimization problem similar to the one considered in [2], it requires personalized offers and individual consumption forecasts; the energy production cost is optimized over the discount values, each of which is offered to a given fraction of the population,

  2. the optimized mechanism takes full advantage of personalized offers and consumption forecasts by minimizing the cost over both the discount values and the population fractions to which the discounts are offered,

  3. the robust mechanism relies on personalized offers, but does not need individual consumption forecasts,

  4. finally the broadcast mechanism (analogous to that in [5]) needs neither of the two features.

Interestingly, contrarily to prior studies, we find that the cost-minimization problems resulting from our DR mechanisms are not convex (even for the base mechanism). Nevertheless, simple heuristics can identify (potential) minima in a reasonable amount of time in realistic scenarios. Then, our numerical results show that the simpler robust and broadcast mechanisms achieve significantly lower cost reductions than the optimized mechanism, which is difficult to implement, but that the gap reduces when the population’s flexibility increases.

The paper is organized as follows. In Sec. 2 we discuss how the macroscopic models considered in [2, 3, 4] hide some implicit assumptions about the user rationality or about the interactions between the provider and the user. We define our microscopic model in Sec. 3 and then describe different DR mechanisms and their corresponding optimization problems in Sec. 4. Finally, we evaluate their performance numerically in a realistic scenario in Sec. 5.

Due to space constraints proofs, examples and some additional numerical results are in the companion technical report [9].

2 Pitfalls when Starting from Macroscopic Models

In this section, we describe in more detail the macroscopic models proposed in the literature for day-ahead price optimization. Consider a finite time horizon discretized in a set of timeslots and a large population of users. The baseline aggregate energy consumption in slot is denoted by .

The energy provider charges a flat rate , but it can offer discount rates to incentivize the users to move some of their consumption so as to reduce the energy production cost. Due to consumption shifts, the actual aggregate consumption in time slot is . Observe that a usual assumption in the literature (including the papers mentioned above) is that the introduction of a DR scheme neither reduces nor increases users’ demand; it merely rearranges users’ consumption in a more cost effective way, so that


We denote the amount of consumption shifted from slot to slot as , and the amount of consumption the users refuse to shift away from as . Then we have

We now start to further detail the model considering some specific assumptions made in previous works. In [2] and [3], the electricity provider offers an energy price discount in each slot . The users are assumed to react to these incentives by shifting a fraction of their baseline consumption from slot to slot ( slots away) according to the following formula:


is called the aggregate sensitivity function and is increasing in the discount and decreasing in the temporal shift , in order to take into account the user discomfort.

The provider selects the vector of discounts

in order to minimize its total cost, equal to the sum of the electricity generation costs and the loss of revenues due to the discounts. In particular the optimization problem considered in [2] is the following:

s.t. (4)

where is the cost of electricity production in slot . Eq. (4) guarantees that discounts are non negative and smaller than the flat rate , so that the money stream goes toward the provider.

As it often happens, the devil is hidden in the details, and in this case in Eqs. (2) and (3). Our first remark is that the cost of lost revenues in Eq. (3) implicitly assumes the possibility to reward only the consumption actually shifted from to , i.e., , but this quantity cannot be directly measured. The actual consumption can be measured, and then can be quantified provided that we have good estimates of the sensitivity function and of the baseline consumption

. Let us assume for a moment that

is known from historical data and that the aggregate baseline consumption may be predicted with a reasonably high level of accuracy on a large set of users. Then it seems possible to solve the macroscopic problem in Eqs. (3) and (4), but we need to consider also what should happen at the microscopic scale. While the estimates for the aggregate baseline consumption can be adequately precise, finally the billing is done at the user’s granularity and each user expects to receive the price discount corresponding to the energy consumption she actually moved. If the energy bill’s reduction does not correspond to her forecast, the user is likely to opt out of the program (in particular if she has experienced underpayments) or to reduce her efforts and milk occasional discounts. It appears then that Eq. (3) implicitly requires very precise predictions of individual consumptions.

We now observe that the form of the sensitivity function in Eq. (2) indicates that the amount of energy shifted from to depends on the discount but not on the other discounts. We can then ask ourselves which individual decisions may lead to this aggregate behavior, an issue ignored both in [2] and [3]. As long as a rational individual is offered two different discounts and , it seems natural that her decision to move some consumption from to or from to or to keep it in will take into account both the discounts. To stress the point, consider a case when both and are positive, but moving the consumption from to is both less inconvenient (i.e., ) and more rewarding (i.e., ). There is then no reason why the user would move consumption to . The conclusion is the same for all the users and then we should have at the aggregate level, in contradiction with Eq. (2). We can then conclude that the expression of the sensitivity function in Eq. (2) is not suited to model the situation when a user is offered two or more rewards, but it can capture the case when the user decides between moving from to in exchange of a discount or staying in . If every user is offered a single discount to move to a given slot, but different users can receive different offers, then Eq. (2) can reasonably describe the macroscopic effect of such personalized offers. The details are described in Sec. 4.1, here we only highlight that Eq. (2) requires then that the electricity provider i) calculates an offer for each user, ii) communicates individually to the user, iii) considers the individual offer when billing the user. This is clearly more demanding than simply advertising to the whole population the same set of discounts.

We observe that the equivalent sensitivity function considered in [4] poses similar problems. Using our notation, we have , where is the net utility a user gets by consuming electricity in slot and can be a function of the timeslot itself and of the discount . This formula tries to capture the effect of the whole set of discounts, but it is not clear again what is the underlying user’s model: if slot has a larger utility than slot (), why should the user consume in ?

3 Starting from a Microscopic Model

In the previous section we made the point that, while aggregate population models may be convenient, it is necessary to explicitly consider the microscopic level: how the user takes the decisions and how the provider and the user are supposed to interact. In this paper we follow the opposite path in comparison to the existing works mentioned: we move from the microscopic level to the macroscopic one. In particular, in this section, we start from a clear model of rationality for the single user and then move to describe how aggregate quantities can be derived.

Each user has a baseline energy consumption that leads to the aggregate baseline consumption for . We assume in what follows that users are homogeneous, i.e.,


In [9] we show how the DR mechanisms perform when this assumption does not hold.

User is characterized by a private type where indicates the discomfort due to shifting one unit of consumption from timeslot to timeslot . We assume that discomforts are expressed in monetary units; and that, ,


i.e., there is a strictly positive discomfort if and only if consumption is shifted from its original timeslot. The provider does not know the private type of each user : from its point of view, each discomfort is drawn from a known, continuous distribution on (where possibly ). Discomforts of distinct users are independent but note that we do not assume that, for a given user, the discomforts are mutually independent.

3.1 Rational Users

We assume that a user simply chooses the option that maximizes her utility. In particular let be a set of timeslots the user could move the baseline consumption to in exchange for different discounts . The set pair defines the offer user receives for timeslot . The set of options includes the possibility to keep the consumption in , i.e., . A rational user maximizes her utility by scheduling her consumption to a timeslot


We assume that if two or more timeslots are equally palatable, the whole consumption is shifted to only one of them, picked at random with equal probability.

3.2 Aggregation

We observe that the quantities in Eq. (7

) are random variables, then two different users could take different decisions while confronted with the same offers. The aggregate consumption

, for , would then be a random variable. Here we assume (as it is implicit in the other works) that we always work with large sets of the population so that the variability can be neglected by approximating actual random quantities with their expected values. In particular, if a subset containing a fraction of the population receives an offer , the corresponding consumption shifted from to a time slot , denoted as will be


if the probability that a user has two or more equally palatable timeslots is zero. When discomforts are continuous random variables (as we consider in this paper), this is always the case if each user receives only one offer (the first three mechanisms introduced below) or if the discomforts

are mutually independent. In Sec. 4.4, we discuss how Eq. (8) should be modified if this probability is not zero. We denote simply as .

4 DR mechanisms

Under different assumptions on the provider’s capabilities, we introduce different demand response mechanisms based on the microscopic model above, which are therefore practically implementable. We introduce and study the corresponding optimization problems.

We start by the base mechanism that leads to the same aggregate optimization problem considered in [2, 3].

4.1 Base mechanism

This mechanism requires that the energy provider can manage personalized offers to its customers and moreover that it has perfect knowledge (or very precise estimates) of the baseline consumption of each user.

The population is segmented into disjoint subsets , for , respectively including a fixed fraction of the population. Each user in is simply offered to move her baseline consumption in slot () to slot in exchange for a price discount .

The total consumption that is shifted from to is then

as it can be obtained from Eq. (8), taking into account that in this case and . We observe that the probability appearing on the right-hand side only depends on the reward and on the random variable . If the discomfort is only a function of the temporal distance , then the sensitivity function (the ratio of people who move from to ) has the same properties than in [2], in particular:

where for we have made explicit the only variable it depends from.

As we discussed in Sec. 2, because the provider knows exactly the consumption shifted from each user, it can formulate the optimization problem (3-4). In [2] it is stated that the problem is convex if i) the productions costs are continuous piecewise linear and increasing and ii) the discomfort distributions are continuous and concave. We show in [9] that this is not the case by providing a counterexample. Stronger hypotheses are required for the problem to be concave, as for example the linearity of the discomfort functions.

In particular in [2] the numerical evaluation considers

that leads us to consider


This particular expression for can be obtained if is a uniform random variable with support in . The numerical results for the base mechanism in Sec. 5 are obtained considering the same expression for the fractions .

Due to the non-convexity of the optimization problem (3-4) we cannot use one of the classic algorithms for convex optimization. For the results shown in section 5

we have adopted instead a multi-start approach: we have generated random starting points uniformly distributed in the problem domain and we have run per each point a descendent algorithm which converged on a local minimum; the optimal offers are therefore those returning the smallest cost among these minimizers. This approach does not guarantee convergence to the global optimum but its reliability can be improved by increasing the number of starting points.

4.2 Optimized Mechanism

We have now understood which DR mechanism can lead to the optimization problem (3), but now that we look at its implementation at the microscopic level and the need for personalized offers, some specifics of the base mechanism look arbitrary and unjustified. For example, given that discounts are not broadcast but each user receives an individual offer, why should the discounts offered to the two disjoint sets of users and be equal to the same value ? It is clear that the energy provider can further reduce the cost if it can independently choose and . Moreover, there is no reason to think that the size of the sets should be fixed, the fractions can also be optimization variables.

We allow the provider to take advantage of these additional degrees of freedom that—we repeat—do not impose any additional requirement to the system. We call this new DR mechanism optimized. The load

rescheduled from to is now and the cost minimization problem becomes:

s.t. (11)

Eq. (11) guarantees that discounts are non negative and smaller than the flat rate , Eq. (13) is a consequence of the fact that each user receives at most one offer for its baseline consumption in a given slot.

The optimization problem (10-13) can be solved with the same heuristic proposed for problem (3-4).

4.3 Robust Mechanism

The optimization problems (3-4) and (10-13) assume that the provider has perfect knowledge of each user’s baseline consumption, so that it can correctly identify the consumption shifted and reduce accordingly the energy bill. This assumption is probably unrealistic. If the provider does not have such capability, then it can offer the user a discount for all the consumption in a given timeslot and not just for the consumption moved to . The population is then divided into subsets , each containing a fraction of the users. All users in receive one and only one offer: they are encouraged to shift their consumption from any timeslot in the time horizon to timeslot and they get the discount for all the electricity consumed in , including the one originally in .

We call this scheme robust, because it does not rely on estimates of individual consumption. It is clearly simpler than the previous two, because the provider needs only to measure the amount of consumption in for the users who got the offer and to bill them accordingly.

The load shifted from to is . Note that users in have no interest to move their baseline consumption away from , then . The robust mechanisms leads to the following optimization problem:

s.t. (15)

Note that in Eq. (14) the first sum includes also because all the final consumption in from the users in is paid at a discounted price. The term does not appear in Eq. (3) and Eq. (10).

The optimization problem (14-17) can be solved with the same heuristic proposed for problem (3-4).

4.4 Broadcast Mechanism

In the three mechanisms introduced above, the provider makes personalized offers to users in selected fractions of the population. This may not always be possible (due to the complexity it introduces for instance in billing) or desirable (for perceived fairness issues). Our last mechanism, which is the simplest (in its definition), does not assume personalized offers. The provider selects a single vector of discounts for every time slot and broadcasts these discounts to all users (hence the name broadcast mechanism). Users then re-arrange their demand and pay the discounted price for their demand in each slot (hence this mechanism also does not rely on the need to estimate shifted demand).

As explained in Sec. 3.1, each individual user moves her demand from slot to a slot (potentially itself) that maximizes her net utility (discount minus discomfort). Recall that if several slots give equal net utility, the user chooses one of them randomly.

Until now, we have not made any assumption on the possible correlation of a given user’s discomforts. This is because, in the previous three mechanisms, each user was receiving only one offer.In the broadcast mechanism, each user has several offers to compare to decide on his new demand schedule, we therefore need to describe the discomfort correlations.

Let us consider now the particular case when two slots, say and may appear equally attractive to a user, i.e. . If we assumed that, for each user, the discomforts were mutually independent, this event would have probability zero according to our assumption on , and therefore it would not appear at the aggregate level. As a result, the aggregate demand moved from to would be , where


However, rather than making the above independence assumption, we prefer to assume that the discomforts have the form , where is a constant independent of the user and

is a random variable with concave Cumulative Distribution Function (CDF)

. This model describes a symmetric delay sensitivity of users (users are indifferent between moving two hours earlier or two hours later) while keeping the flexibility of users having a different flexibility of demand of different times (since and are indexed by the origin timeslot ); but it also introduces correlations between the discomforts of a user. As a result, the fraction of demand shifting from to is


rather than (18). The denominator in (19) accounts for cases when a slot other than (which has to be ) gives equal net utility for all users. The broadcast mechanism then leads to the following optimization problem:

s.t. (21)

Unfortunately, due to indicator function in Eq. (19), the cost function (20) of the broadcast mechanism is not continuous, even in very simple scenarios with continuous production costs [9]. Discontinuity arises also in the macroscopic model in [5], but it seems to have been ignored.

In practice, we solve problem (20-21) using the same heuristic proposed for the previous problems, but we work on a continuous and smooth approximation of the cost function.

4.5 Ranking DR mechanisms

In [9] we prove that the optimized mechanism always performs better than the base and robust mechanisms, but the ranking cannot be extended further.

5 Numerical Results

In this section we evaluate the performance of the different DR mechanisms in the realistic scenario considered in [2] and based on energy data about the Ontario province in Canada. In particular, the baseline consumption and the cost functions are estimated from the IESO energy portfolio [8], consisting of nuclear plans, hydro gas powered stations and renewable and from typical costs associated to these energy sources. We considered the flat rate and the timeslot-independent piecewise-linear cost function with derivative: for , for and for , where and represent respectively the base to intermediate load capacity and intermediate to peak load capacity. We assume that discomforts take the form , where is an exponential random variable with cdf . is a parameter representing the population’s flexibility. The larger it is, the smaller (in a stochastic order sense) the discomfort of the users to shift their consumption.

In Fig. 1, we report the cost savings of the DR schemes, normalized to the initial cost, for four different values of the parameter: , , , . The dashed line represents the saving which could be achieved if users’ demand could be rearranged at the provider’s will without providing any discount (we indicate it as the dictatorial solution).

Figure 1: Cost savings normalized to the initial cost, for various flexibility parameters .

Consistently with the results in Sec. 4.5, the optimized mechanism returns larger savings than the robust and the base ones. Interestingly, the robust mechanism performs consistently better than the base one despite the fact that it does not require the ability to estimate the demand shifted and it therefore “wastes” some discount by giving it to demand that was already scheduled in a given timeslot in the baseline demand. Moreover, as the population flexibility increases, the savings gap between the optimized scheme and the robust mechanism reduces, the latter being effectively close to exploiting all the population’s flexibility.

In Fig. 2, we focus on the case and analyze the components of the cost for each DR mechanism. Fig. 2 confirms that the optimized scheme provides the largest savings as it can minimize the production cost while paying the smallest amount of discounts. We indicate with wasted discounts the amount of discounts paid to consumption that would in any case have been scheduled in that timeslot. The base and optimized mechanisms do not waste any discount, while the robust mechanism and the broadcast scheme do, as they provide the discount to all the electricity consumed in , including the part of that remains in .

Figure 2: Analysis of the components of the cost savings. All the quantities are normalized to the initial cost.

6 Conclusions

In this paper, we have shown that macroscopic descriptions of DR mechanisms can hide important assumptions that can jeopardize the mechanisms’ implementation. For this reason, our proposal moved from a microscopic description that explicitly models each user’s decision. We have then introduced four DR mechanisms with various assumptions on the provider’s capabilities. Interestingly, contrarily to previous studies, we find that the optimization problems that result from our mechanisms are complex and can be solved numerically only through a heuristic. Moreover, our results show that the performance of DR mechanisms under reasonable assumptions on the provider’s capabilities are significantly lower than those suggested by previous studies, but that the gap reduces when the population’s flexibility increases.

7 Acknowledgements

This work was partly funded by the French Government (National Research Agency, ANR) through the “Investments for the Future” Program reference #ANR-11-LABX-0031-01.


  • [1] M.H. Albadi, E.F. El-Saadany, “A summary of demand response in electricity markets,” Electric Power Systems Research, Volume 78, Issue 11, November 2008, Pages 1989-1996
  • [2] C. Joe-Wong, S. Sen, S. Ha, and M. Chiang. “Optimized Day-Ahead Pricing for Smart Grids with Device-Specific Scheduling Flexibility.” IEEE J. on Selected Areas in Communications, vol. 30, no. 6, pp. 1075–1085, 2012.
  • [3] Li Pan, Guan Xiaohong, Wu Jiang and Wang Dai, “Pricing Strategy for Device-Level Demand Response in a Microgrid.” In Proceedings of the 33rd Chinese Control Conference (CCC), pp. 7579–7584, July 2014.
  • [4] S. Subramanian, S. Ghosh, J. R. M. Hosking, R. Natarajan, X. Zhang, “Dynamic Price Optimization Models for Managing Time-of-Day Electricity Usage,” In Proceedings of the 2013 IEEE International Conference on Smart Grid Communications (SmartGridComm), pp. 163–168, 2013.
  • [5] W. Yang, R. Yu, M. Nambiar, “Quantifying the benefits to consumers for demand response with a statistical elasticity model,” IET Generation, Transmission Distribution, vol. 8, no. 3, pp. 503–515, 2014.
  • [6] L. Song, Y. Xiao, M. van der Schaar, “Demand Side Management in Smart Grids Using a Repeated Game Framework,” IEEE J. on Selected Areas in Communications, Vol. 32, no. 7, pp. 1412–1424, 2014.
  • [7] J. Gorski, F. Pfeuffer, and K. Klamroth. “Biconvex sets and optimization with biconvex functions: a survey and extensions.” Math Meth Oper Res, vol. 66, no. 3, pp. 373–407, 2007.
  • [8] IESO Ontario, “Market data,” Sep. 2011, http://www.ieso.ca/imoweb/marketdata/marketData.asp.
  • [9] A. Benegiamo, P. Loiseau, G. Neglia, “Dissecting Demand Response Mechanisms: the Role of Consumption Forecasts and Personalized Offers,” Inria Research Report no. 8881, 2016