Gibbs sampling for game-theoretic modeling of private network upgrades with distributed generation

by   Merlinda Andoni, et al.

Renewable energy is increasingly being curtailed, due to oversupply or network constraints. Curtailment can be partially avoided by smart grid management, but the long term solution is network reinforcement. Network upgrades, however, can be costly, so recent interest has focused on incentivising private investors to participate in network investments. In this paper, we study settings where a private renewable investor constructs a power line, but also provides access to other generators that pay a transmission fee. The decisions on optimal (and interdependent) renewable capacities built by investors, affect the resulting curtailment and profitability of projects, and can be formulated as a Stackelberg game. Optimal capacities rely jointly on stochastic variables, such as the renewable resource at project location. In this paper, we show how Markov chain Monte Carlo (MCMC) and Gibbs sampling techniques, can be used to generate observations from historic resource data and simulate multiple future scenarios. Finally, we validate and apply our game-theoretic formulation of the investment decision, to a real network upgrade problem in the UK.



page 1

page 2

page 3

page 4

page 5

page 6


Game-theoretic modeling of curtailment rules and network investments with distributed generation

Renewable energy has achieved high penetration rates in many areas, lead...

Techniques for proving Asynchronous Convergence results for Markov Chain Monte Carlo methods

Markov Chain Monte Carlo (MCMC) methods such as Gibbs sampling are findi...

Exploration of Gibbs-Laguerre tessellations for three-dimensional stochastic modeling

Random tessellations are well suited for the probabilistic modeling of t...

A table of short-period Tausworthe generators for Markov chain quasi-Monte Carlo

We consider the problem of estimating expectations by using Markov chain...

Sequential Gibbs Sampling Algorithm for Cognitive Diagnosis Models with Many Attributes

Cognitive diagnosis models (CDMs) are useful statistical tools to provid...

Gibbs Sampling with People

A core problem in cognitive science and machine learning is to understan...

A comprehensive study of Game Theory applications for smart grids, demand side management programs, and transportation networks

Game theory is a powerful analytical tool for modeling decision makers s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Renewable energy sources (RES) play a key role in the climate change mitigation agenda. RES are variable, depend on weather patterns and are difficult to predict, hence raise technical challenges regarding network management. Moreover, grid infrastructure is often inadequate to support RES development, especially in the area of distribution networks. For instance, often RES projects are clustered in remote areas of the grid, where planning approval may be favourable and renewable resources abundant. Typically, in the UK, such areas are windy islands with constrained connections to the main grid. In these areas, technical limitations and imbalance of renewable supply to demand, often result in RES curtailment, i.e. the energy that could have been generated is wasted as the system cannot absorb it or transfer it where required.  Curtailment can affect implementation of future RES projects, result in lost revenues and instigate socio-economic challenges, especially when local community-owned projects are involved.

Typically, RES generators are granted firm connections to the grid and receive compensation for incurred curtailment, the cost of which is eventually borne by all system users. However in several occasions, RES generators are offered interruptible, non-firm connections, as an alternative to expensive or time consuming reinforcements. The so-called flexible commercial arrangements are increasingly offered by network operators, as an alternative, and are creating a shift in network access rules. A detailed review on flexible commercial arrangements can be found in [9, 8]. As shown in [3], arrangements that are fair and equally share curtailment and access among generators, can maximise the generation capacity built at a certain location and minimise discouragement for future investors.

Solutions to reduce curtailment include demand side management, energy storage or smart grid techniques such as Dynamic Line Rating (DLR) or real-time monitoring of the thermal state of the lines [11] and Active Network Management (ANM), i.e., the automatic control of the power system by control devices and data that allow real time operation and optimal power flows [1]. The long term solution, however, is network upgrade. Anaya & Pollit (2015) compared smart interruptible connections to traditional grid reinforcements in [1]

.  It is estimated that, by 2030 in the US alone, up to $2 trillion will be required for network upgrades 

[5]. As grid expansion is expensive, it is desirable to provide incentives to private parties for taking part in such investments.  Private network upgrades have been studied in several works [10, 13].  However, they raise the question for system operators of defining the framework within which these private lines are incentivised, built and accessed by competing generators.

New private lines constructed, often follow a ‘single access’ principle, i.e. lines for sole-use that suffice only to accommodate the RES capacity of each project. However, as shown in [3], it is possible for system operators to encourage RES generators to install larger capacity lines under a ‘common access’ principle, i.e. a private investor is licensed to build a line only if it grants access to smaller generators, which are subject to transmission charges. In these settings, curtailment and line access rules can play a significant role in the resulting grid expansion. Crucially, this leads to a leader-follower or a Stackelberg game between the line investor and local generators respectively. Stackelberg games in network upgrades and RES settings have been presented in [14, 4]. Our previous work was one of the first to introduce Stackelberg game formulations in settings that combine network upgrades, curtailment and line access rules,  however the model presented in [3] did not take into account stochastic generation and demand required for equilibrium estimation. Subsequent work [2] improved by utilising real data, however, followed a one-shot, single scenario approach. We build on this work by developing a principled framework, based on game-theoretic and state-of-the-art sampling techniques, i.e. Markov chain Monte Carlo (MCMC). Several authors used MCMC for modelling of wind speeds or wind power outputs [12, 15].  Our framework allows modelling multiple renewable investment scenarios that reduce the uncertainty of future generation and demand.  In more detail, the main contributions of the work are:

  • We develop a new methodology that generates observations from renewable resource data. While historic data, such as wind speeds may be available, they might have considerable gaps and joint distributions cannot be expressed in simple, closed-form equations. For this reason we develop a MCMC methodology (Gibbs sampling) that can draw samples from available data and run multiple scenarios of potential futures.

  • We establish a methodology that can determine optimal generation capacity investments through use of real demand and wind speed data. This work is one of the first to combine Stackelberg equilibria to a large-scale realistic game with MCMC techniques. Our model designates players’ actions, depending on RES output correlation and expected curtailment, and studies the cost parameters effects on the equilibrium of the game.

  • Previous work has been extended to account for varying demand data instead of an average approach followed in both [3, 2]. The model is applied and validated in a real network upgrade problem in the UK.

Section II of the paper presents the theoretical formulation of the game, Section III introduces Gibbs sampling, Section IV demonstrates the methodology based on a real-world case study, Section V shows the results and Section VI concludes.

Ii Stackelberg game model description

Consider a simplified two-node network: location A is a net consumer (area of high demand) and location B is a net producer (area of high wind resource). Moreover, consider two players: a line investor, who can be merchant-type or a utility company and is building i) the AB interconnection and ii) renewable capacity at B, and a local player representing all local RES generators located at B, and who builds renewable capacity equal to . This second player can be thought of as investors from the local community, who do not have the technical/financial capacity to build a line, but may have access to cheaper land, find it easier to get social approval to build turbines etc., hence may have lower costs for RES deployment.

Building the line will elicit a reaction from local investors. Crucially, the line investor has a first mover advantage in building the grid infrastructure, which is expensive, technically challenging, and only few investors (e.g. DNO-approved) have the expertise and regulatory approval to carry it out. From a game-theoretic perspective, this is a bi-level Stackelberg game, between the leader (line investor or first player) and follower (local generators or second player). We assume that players’ actions are driven by profit maximisation criteria. The profit functions of players can be expressed as in [2]:


In these equations, represents player’s expected energy produced over the project lifetime, if no curtailment occurred, while is the energy lost through curtailment. Line cost is estimated as , where is the cost of building the line (or initial investment) and the cost of operation and maintenance. The monetary value of the power line is proportional to the energy flowing from B to A, charged for local generators and ‘common access’ rules with transmission fee per energy unit transported through the line.  Moreover, the cost of expected generation per unit is defined as , where is the cost of building the plant and the operation and maintenance costs. Finally, the energy generated by a RES unit is sold at a constant generation tariff price, equal to .

As seen in Eq. (1), the line investor has two streams of revenue, the self-produced energy and the energy produced by local generators and transported through the line. The costs of the line investor relate to installing and operating the RES capacity () and to building the power line (). Similarly from Eq. (2), the profits of the local generators depend only on the energy produced (generation cost ) and transmitted through the line at a charge of .

The research question our model tries to answer is ‘How to determine the optimal generation capacities built by the two players, so that profits are maximised?’  To answer this question, the profit equations, Eq. (1) and Eq. (2), need to be expressed in terms of generation capacities.

Following the analysis presented in [2], expected generation and curtailment can be expressed as functions of the players’ generation capacities. If is the per unit power generated by player, then this represents a stochastic variable that depends on the wind speed distribution, and is equal to , where is the actual power output of generator and its rated capacity. If is the expected power generated at time interval , then the total energy generated for duration equal to the project’s lifetime is . Similarly, the total energy curtailed is , where is the expected power curtailed at time step .

The resulting curtailment depends on wind resources at location B and demand at A, denoted as . Curtailment events happen when . Players’ generating plants are located at neighbouring locations at B, therefore experience correlated wind speeds. The stochastic variables and

, follow a joint probability distribution function

and expected curtailment at time interval can be expressed as (detailed analysis shown in [2]):


Crucially, Eq. (3) shows that curtailment depends on both players’ strategies i.e. the generation capacities built. Curtailment expressions for each player under a ‘common access’ regime can be reasonably can be approximated by . This concludes the expression of profit equations as functions of players’ rated capacities.

Optimal capacities installed are determined in the equilibrium of the game, which is found by backward induction. The line investor or leader assesses and evaluates the second player’s reaction, in order to determine his strategy (i.e. ) and influence the equilibrium price. The leader estimates the follower’s best response, given his own capacity :


Next, the leader estimates which solution from the set of the local generators’ best response maximises his own profit:


In other words, the leader moves first by installing their own capacity. In the second level, followers respond to the capacity built, as anticipated by the leader. The equilibrium of the game satisfies both Eq. (4) and Eq. (5) and is given by the notion of the subgame perfect equilibrium.

In practical settings the joint distribution of stochastic renewable resources is often unknown, but historic data may be available. In addition, due to the interdependency in resulting curtailment and multiple parameters a nice closed-form solution of the game cannot be found or expressed analytically. In [2] we presented an empirical algorithm that utilises directly real data and approximates the solution of the game following a one-shot approach. In addition, data may experience important gaps. In this paper we show how we can utilise real data to simulate scenarios that approximate the real distribution with a state-of-the-art MCMC technique.

Iii Gibbs sampling

Markov chain Monte Carlo (MCMC) is a class of methods for simulation of stochastic processes. Gibbs sampling can be thought of as a particular case of the Metropolis-Hastings algorithm used for MCMC [6]. The Gibbs sampler uses the conditional distributions as proposal distributions with acceptance probability equal to 1 [7] and can be easily implemented in various applications. Using this technique we can generate, from historic data, observations that are dependent. Wind data samples from the project locations form a Markov chain (MC). We can experiment with the length of the chain or sampling size and we can repeat the process for multiple MCs or number of realisations . In practice, and need to be determined in such way that the resulting MC converges to the real distribution, is ergodic and computationally efficient. Ergodicity means that all possible states of the MC can be visited and are independent of the starting state [7]. The methodology applied is described in detail in the next sections with the help of the Kintyre-Hunterston case study.

Iv Case study analysis

Iv-a Kintyre-Hunterston link

Fig. 1: Kintyre-Hunterston project map11footnotemark: 1

The case study analysis is based on a real grid reinforcement project in the UK that links the Kintyre peninsula to the Hunterston substation on the Scottish mainland (see Fig. 1). Kintyre is a region that has attracted vast renewable investment, predominantly wind generation, resulting in the necessity of a newly built power line, which provided space for of additional renewable capacity.

max width= Sample size   WCI ME   WCI ME    WCI ME

TABLE I: Results for realisations and an increasing number of observations

max width= Realisations   WCI ME   WCI ME    WCI ME

TABLE II: Results for sampling size and an increasing number of realisations

Following the analysis described in Section II, we assume that the demand region or Location A is Hunterston and location B is the geographical region covering the Kintyre peninsula. The line investor and local generators install generation capacities at different sub-regions of B. Two weather stations were selected by the UK Met Office database222, the station with ID 908 located in the Kintyre peninsula (wind farm of line investor) and with ID 23417 located in Islay (wind farm of local generators), with a distance between them of . These weather stations provide data over a common period of 17 years (1999–2015). Demand data used in simulations are based on real UK National Demand data 333 in the time period of 2006–2015. UK demand data are normalised to represent a lower local demand. More details on the case study and data processing can be found in [2].

Literature in wind forecasting commonly uses Weibull distributions for the representation of actual wind distributions. However, the joint probability distribution of the wind speed (and of the players’ power outputs), exhibits correlation and in practice is not known. If there are sufficient wind speed measurements for both players locations, then the joint probability distribution can be approximated directly from the available historic data. The method described in the following section can be used to draw observations from available data and simulate different scenarios. The technique can generate large datasets as required for the intended analysis.

Iv-B Gibbs sampling applied

1:,, wind speed 1,2, power demand
2: number of samples
3:, historic data
4: wind distribution from data
5: demand cond. distrib. on mean wind
7: initialise
14:return ,
Algorithm 1 Gibbs Sampling

We apply Gibbs sampling to the joint bivariate distribution of wind speeds at the players’ locations. Players’ wind speeds at each time form a MC. The methodology is described in Alg. 1. From available historic data, we create a joint distribution table of wind speeds at the players’ locations. For every possible wind speed of first player, we record the subset of

wind speed, and vice versa (Line 4 in Alg. 1). This represents the conditional distributions. In practice,the probabilities for certain combinations of wind speeds can be low (e.g. it is unlikely to have extremely high wind at one location and low wind speed to a proximal location), therefore some subsets can be sparse, either because some of observations represent rare events or due to correlation. We overcome this difficulty by merging sparse bins of rare events and outliers, and ensure ergodicity of the MC. The MC is initialised by randomly selecting a sample from the joint distribution table (Line 7 in Alg. 1). Each iteration step involves replacing the value of one variable by a value selected randomly by the conditional

. In addition, demand is randomly selected by the conditional distribution of demand over the average wind speed (Line 11 in Alg. 1). The procedure is cycled through the variables forming samples of , . To ensure that the MC converges, we run Alg. 1 for several sampling sizes (small, moderate, large) and repeat the procedure for realisations. Results are shown in Tables I and II

. Columns represent the sample mean, standard deviation of the sample mean, width of the 95% confidence interval (WCI) and maximum error (ME) from mean of historic data, i.e.

, and

. As the sampling size increases the sample mean follows a normal distribution and the standard deviation decreases, as expected by the central limit theorem (Table 

I). We perform the same analysis for an increasing number of realisations that use a different starting point (Table II). We also adopt a burn-in or warm-up period of 20% of samples to make sure that our results are independent off the starting state [7]. The results show that MC converges to the distribution from data and that a large is required but can be chosen to be relatively small. For this reason and driven by computational limitations, for the estimation of the Stackelberg equilibrium, we selected , and burn-in of samples.

Iv-C Stackelberg equilibrium estimation

Wind speed data generated by the Gibbs sampler are used to estimate the per unit power output of wind generators. Estimation is based on a generic power curve444Parameters derived by a Enercon E82 wind turbine:

and a sigmoid function approximation (see Alg. 2 Line 4). Players’ strategies are the capacities they can install. The maximum feasible solution for a single player was set equal to

and the incremental capacity to . For every possible combination of the rated capacities installed , we estimate the power generated and curtailed for each player on an hourly basis. Next, we estimate the aggregate power generated and curtailed by each player as the summation of data points. The procedure is described in Alg. 2.

1: max rated capacity in search space
2: strategy space i=1,2 player
3:, power curve sigmoid parameters
4: generation i player
5:for all do
6:     for all do      
7:          residual demand
8:         if  then no curtailment
10:         end
11:          curtailment gen 1
12:          curtailment gen 2      
13:     end
15: total gen i
16: total curt 1
17: total curt 2
Algorithm 2 Generation & Curtailment Estimation

For several cost parameters and feed-in tariff price , we estimate the profits as defined in Eq. (1) and Eq. (2). For every possible , we find the capacity that maximises the follower’s profits (follower’s best response). From this set of solutions, the leader selects the one that maximises its own profit i.e. (leader’s best response). The equilibrium of the game is given by the pair , which satisfies best response functions as described in Alg. 3.

1: feed-in tariff, transmission fee
2: cost of line
3: i player’s generation cost
4:for all do
5:     for all do      
9:     end
11:for all do best response gen 2
15: best response gen 1
Algorithm 3 Profit & Stackelberg Equilibria

V Results and discussion

The methodology described in previous sections was followed to run several experiments. Fig. 2 shows the optimal rated capacities built by the players at the equilibrium of the game for realisations. Recall here that every realisation represents a completely different MC generated. The results are satisfactory and show a range in the estimated solutions for optimal rated capacities. Similar results were observed for the optimal profits derived.

Fig. 2: Optimal generation capacities of each player for several realisations (, , and )

Moreover, we study how the equilibrium results depend on varying cost parameters. Fig. 3 shows the dependence on line investor’s cost (first column), local generators’ cost (second column) and the transmission fee (third column). We assume that both players can sell the energy generated for . For each scenario, the key parameter varies, while other parameters remain fixed (Scenario 1: , and , Scenario 2: , and and Scenario 3: , and ). The results in Fig. 3 show the average equilibrium solution and min-max solutions, found for realisations of the simulation procedure.

Fig. 3: Rows (1) and (2) show generation capacity built and profits at Stackelberg equilibrium, respectively, column (1) shows dependency on generation cost of line investor, (2) on on generation cost of local generators and (3) on transmission fee

In all sets of scenarios, the total capacity installed by all players decreases as the tested parameter value increases. Each player installs less capacity as their generation cost increases, while the other player benefits by increasing their capacity. The cost of local generators has a larger impact on the capacities installed for both players, as shown by comparing the first to the second column. Profits have similar behaviour to the optimal rated capacities, but local generators face the additional cost of transmission charges. If the followers’ generation cost is much lower than the line investor’s (assuming for example that local generators might have access to cheaper land), the line investor needs to charge a high transmission fee to have positive earnings. On the contrary, if the leader’s cost is much lower, the generation capacity will mostly be installed by the line investor, as there is no room for profitable investment from local renewable producers. As shown in Scenario 3, or is the minimum value of transmission charges that allows profit for the line investor. Similarly, if the transmission fee is set too high, then it is not profitable for local investors to invest in renewable generation. As is set by the system regulator, the methodology can be useful to determine a feasible range of charges that allows both transmission and generation investments to be profitable.

Vi Conclusions & Future Work

In this work we show how privately developed network upgrade for DGs can lead to a leader-follower game between the line and local investors. Curtailment and line access rules play a key role in the strategic game, the equilibrium of which determines optimal generation capacities and their profits.  Settings where this model can be applied include numerous locations where demand and generation are not co-located. When real historic data is available, we can use MCMC and Gibbs sampling to simulate multiple future scenarios and reduce the uncertainty of the investment decisions. In the future, we plan to extend the model to multi-location settings and introduce energy storage, which enables using renewable energy to satisfy more of the outstanding demand, and hence reduces curtailment, changing the joint investment game.


The authors acknowledge the EPSRC National Centre for Energy Systems Integration (CESI) [EP/P001173/1].


  • [1] K. L. Anaya and M. G. Pollitt (2015) Options for allocating and releasing distribution system capacity: deciding between interruptible connections and firm DG connections. Applied Energy 144, pp. 96–105. Cited by: §I.
  • [2] M. Andoni, V. Robu, W. Früh, and D. Flynn (2017) Game-theoretic modeling of curtailment rules and network investments with distributed generation. Applied Energy 201, pp. 174–187. Cited by: 3rd item, §I, §II, §II, §II, §II, §IV-A.
  • [3] M. Andoni, V. Robu, and W. Früh (2016) Game-theoretic modeling of curtailment rules and their effect on transmission line investments. In IEEE 2012 PES ISGT Europe, pp. 1–6. Cited by: 3rd item, §I, §I.
  • [4] G. E. Asimakopoulou, A. L. Dimeas, and N. D. Hatziargyriou (2013-Dec.) Leader-follower strategies for energy management of multi-microgrids. IEEE Transactions on Smart Grid 4 (4), pp. 1909–1916. Cited by: §I.
  • [5] P. Bronski, J. Creyts, M. Crowdis, S. Doig, J. Glassmire, L. Guccione, et al. (2015) The economics of load defection: how grid-connected solar-plus-battery systems will compete with traditional electric service-why it matters, and possible paths forward. Cited by: §I.
  • [6] C. Fox and A. Parker (2014)

    Convergence in variance of Chebyshev accelerated Gibbs samplers

    SIAM Journal on Scientific Computing 36 (1), pp. A124–A147. Cited by: §III.
  • [7] C. J. Geyer (1992) Practical Markov chain Monte Carlo. Statistical science, pp. 473–483. Cited by: §III, §IV-B.
  • [8] L. Kane and G. Ault (2014-05) A review and analysis of renewable energy curtailment schemes and Principles of Access: transitioning towards business as usual. Energy Policy 72, pp. 67–77. Cited by: §I.
  • [9] A. Laguna Estopier, E. Crosthwaite Eyre, S. Georgiopoulos, and C. Marantes (2013) FPP low carbon networks: Commercial solutions for active network management. In CIRED, Stockholm. Cited by: §I.
  • [10] L. Maurovich-Horvat, T. K. Boomsma, and A. S. Siddiqui (2015-05) Transmission and wind investment in a deregulated electricity industry. IEEE Trans. on Power Systems 30 (3), pp. 1633–1643. Cited by: §I.
  • [11] A. Michiorri, R. Currie, P. Taylor, F. Watson, and D. Macleman (2011) Dynamic line ratings deployment on the Orkney smart grid. In CIRED, Cited by: §I.
  • [12] G. Papaefthymiou and B. Klockl (2008) MCMC for wind power simulation. IEEE Trans. on Energy Conversion 23 (1), pp. 234–240. Cited by: §I.
  • [13] A. Perrault and C. Boutilier (2014-05) Efficient coordinated power distribution on private infrastructure. In Int. Conference on Autonomous Agents and Multi-Agent Systems, Paris, pp. 805–812. Cited by: §I.
  • [14] A. H. van der Weijde and B. F. Hobbs (2012-Feb.) The economics of planning electricity transmission to accommodate renewables: using two-stage optimisation to evaluate flexibility and the cost of disregarding uncertainty. Energy Economics 34 (6), pp. 2089–2101. Cited by: §I.
  • [15] T. Wu, X. Ai, W. Lin, J. Wen, and L. Weihua (2012) Markov chain Monte Carlo method for the modeling of wind power time series. In 2012 IEEE PES ISGT Asia, pp. 1–6. Cited by: §I.