Bidding in Smart Grid PDAs: Theory, Analysis and Strategy (Extended Version)

11/19/2019 ∙ by Susobhan Ghosh, et al. ∙ IIIT Hyderabad Tata Consultancy Services 0

Periodic Double Auctions (PDAs) are commonly used in the real world for trading, e.g. in stock markets to determine stock opening prices, and energy markets to trade energy in order to balance net demand in smart grids, involving trillions of dollars in the process. A bidder, participating in such PDAs, has to plan for bids in the current auction as well as for the future auctions, which highlights the necessity of good bidding strategies. In this paper, we perform an equilibrium analysis of single unit single-shot double auctions with a certain clearing price and payment rule, which we refer to as ACPR, and find it intractable to analyze as number of participating agents increase. We further derive the best response for a bidder with complete information in a single-shot double auction with ACPR. Leveraging the theory developed for single-shot double auction and taking the PowerTAC wholesale market PDA as our testbed, we proceed by modeling the PDA of PowerTAC as an MDP. We propose a novel bidding strategy, namely MDPLCPBS. We empirically show that MDPLCPBS follows the equilibrium strategy for double auctions that we previously analyze. In addition, we benchmark our strategy against the baseline and the state-of-the-art bidding strategies for the PowerTAC wholesale market PDAs, and show that MDPLCPBS outperforms most of them consistently.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 11

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Auctions are mechanisms which facilitate buying and selling of goods or items amongst a group of agents. Double auctions are prevalent when both the sides of a market actively bid. For example, in the New York Stock Exchange, opening prices are determined using double auctions parsons2011auctions. In smart grids, multiple power generating companies and different distributing agencies (brokers) trade electricity in the wholesale markets using double auctions.

In this work, we focus primarily on electricity markets. In July 2019, approximately 1.2 Billion Euros worth electricity was traded in Nord Pool alone, with 52% of the volume being traded using APIs nordpool. Any small improvement in cost optimization by deploying better bidding strategies can lead to significant improvements in the profits of the distributing agencies. Motivated by this, we take up a formal game-theoretic approach in this work for devising bidding strategies.

Typically, for double auctions, clearing price and payment rules differ from market to market. Equilibrium analysis of double auctions has been explored extensively with different payment and clearing price rules wilson1992strategic. Specifically, for -double auctions, satterthwaite1989bilateral (satterthwaite1989bilateral) proved the existence of multiple non-trivial equilibria for . They also focused on a class of well-behaved equilibria, by making generalist assumptions on buyer’s and seller’s bidding strategies. Our focus in this paper is Average Clearing Price Rule (ACPR) based Periodic Double Auctions (PDAs), commonly used in smart grids (Power TAC ketter20172018). In ACPR, the clearing price set as the average of last executing bid and last executing ask (a special case of -double auction with = 0.5).

For ACPR, chatterjee1983bargaining (chatterjee1983bargaining

) constructed a symmetric equilibrium for the case of one buyer and one seller with uniformly distributed valuations. However, in the vast literature of double auctions, a generic equilibrium analysis for ACPR with more buyers has not been well explored

wilson1992strategic. We take up a double auction with ACPR as a case study. We assume all the agents involved (buyers and sellers) deploy scaling based strategies, and identify the Nash Equilibrium (NE) of the induced game. Researchers have used fictitious play-based convergence to equilibrium (e.g., shi2010equilibrium) in double auctions. However, such strategies are not useful in PDAs when the agents need to place bids in real-time for new auctions. In such settings, we believe scaling based strategies are easy to interpret and implement. The equilibrium analysis of non-linear or other complex forms are analytically difficult to compute; moreover may not be appealing to the real users of these markets.

We characterize NEs for One Buyer and One Seller (OBOS) and Two Buyer and One Seller (TBOS) analytically (Theorem 1 and 2). Given our assumption of scaling based bidding strategies and uniform type distributions, generic equilibrium analysis of double auctions, following ACPR, beyond these settings is challenging. To test our double auction strategies, we take the help of the PowerTAC simulation environment. PowerTAC is a simulation platform that replicates crucial elements of the smart grid, where multiple distributing agencies (brokers) compete across markets to generate the most profit. Note that, the double auctions in PowerTAC; for that matter actually in electricity markets; are PDAs. In PDAs, the market clears multiple times, each after a specific time interval.

Now, if a buyer knows all the bids in a double auction, we argue that it is a best response for the buyer to bid as close as possible to the last clearing bid in order to procure the full required energy (Proposition 1

). However, in reality, buyers never have access to such information. To address this incomplete information, we model the bidding process in PowerTAC PDAs as a Markov Decision Process (MDP), and solve it using dynamic programming and Last Clearing Price (LCP) prediction. Motivated by Power TAC’s fast response time constraints, we propose a PDA bidding strategy

MDPLCPBS (Algorithm 1). Though our MDP formulation is inspired by urieli2014tactex (urieli2014tactex), the novelty lies in the reward, solution, and application to place bids. First, we illustrate that the MDP based strategy indeed achieves the equilibrium strategy characterized for OBOS setting. Then, we conduct different experiments to compare MDPLCPBS with the following strategies: ZI gode1993allocative, ZIP tesauro2001high, TacTex urieli2014tactex, and MCTS chowdhury2018bidding. Our analysis shows that MDPLCPBS outperforms ZI, TacTex, and ZIP in all the cases, and closely matches with MCTS. Simultaneously, we show that it predicts the LCP with minimal error. We used this bidding strategy to great effect during PowerTAC 2018 Finals passghosh Ghosh2019.

In summary, our contributions are as follows:

  • We analytically characterize NE strategies for OBOS and TBOS settings (Theorem 1 and Theorem 2).

  • We propose the best response in a complete information multi-unit double auction.

  • For bidding in PDAs such as PowerTAC, we design an algorithm MDPLCPBS (Algorithm 1). It is based on dynamic programming and LCP prediction.

  • Experimentally, we validate that MDPLCPBS achieves the equilibrium characterized for OBOS setting. Further, we demonstrate its efficacy against state of the art strategies for PowerTAC, and also show that it predicts the LCP with minimal error.

2 Definitions & Background

We first define all the required terms formally.

Definition 1.

(Periodic Double Auction (PDA)) A type of auction, for buying and selling some resource, with multiple discrete clearing periods i.e. clearing after a specific time interval. Potential buyers submit their bids and potential sellers simultaneously submit their asks to an auctioneer. Then the auctioneer matches the bids and asks, and chooses some clearing price, denoted as , that clears the auction wurman1998flexible. The allocation rule determines the quantity bought/sold by each buyer/seller, while the payment rule determines how much each buyer/seller pays/earns for buying/selling that quantity.

Definition 2.

(Last Clearing Bid/Ask (LCB/LCA)) Last Clearing Bid (Ask) of an auction refers to that partially or fully cleared bid (ask) which has the lowest (highest) limit-price. It is referred to as “last clearing” since it is the last bid (ask) to be cleared by the clearing mechanism of the auction.

Definition 3.

(Last Clearing Price (LCP)) Last Clearing Price (LCP) of bids (asks) refers to the limit-price of the Last Clearing Bid (Ask).

Definition 4.

(The k-Double Auction) If a buyer and seller participate in a double auction, and if the sealed bid by the buyer is higher than the sealed bid by the seller, then is given by for some fixed .

Definition 5.

(Average Clearing Price Rule (ACPR)) In a double auction, the clearing price and payment rule is ACPR if the clearing price is given by where is the last executed bid, and is the last executed ask. It is a special case of k-double auction, with .

Consider a game , , where is the set of players, is the strategy set of the player , and for are utility functions.

Definition 6.

(Best Response) Given a game , the best response correspondence for player is the mapping defined by . That is, given a profile of strategies of the other players, gives the set of all best response strategies of player .

Definition 7.

(Nash Equilibrium) Given a game , a strategy profile is said to be a Nash Equilibrium of if, . That is, each player’s Nash Equilibrium strategy is a best response to the Nash Equilibrium strategies of the other players.

Definition 8.

(Markov Decision Process (MDP) puterman1994markov) A Markov Decision Process (MDP) is a tuple given by where is the set of states, is the set of actions,

is the state transition probability function, where

is the probability that action in state at time will lead to state at time , is the reward function, with denoting the reward obtained by taking action in state , and is the discount factor.

PowerTAC

In this work, we focus more on smart grids. The Power Trading Agent Competition (PowerTAC) ketter20172018 environment simulates a smart grid for approximately 60 days, where multiple brokers compete against each other across three markets - tariff, wholesale and balancing market - to generate the most profit. Each broker maintains a portfolio of consumers and producers, and buys and sells energy in the wholesale market. The broker with the highest bank balance at the end of the simulation, wins the game. We use the PowerTAC simulator to benchmark our bidding strategy.

The PowerTAC wholesale market employs PDAs for wholesale market energy trading. The clearing price and payment rule for the PowerTAC PDA, is given by . Three types of entities participate in these auctions - (1) Generating Companies (GenCos), (2) Miso Buyer, and (3) PowerTAC brokers. GenCos place only asks to sell energy, while the Meso Buyer places very low bid prices to buy energy. The PowerTAC brokers are free to place a bid or an ask depending on their requirement, or not place a bid at all.

The brokers can always participate in 24 auctions to trade energy, one auction for each of the next 24 timeslots. Each broker is notified about identity of other brokers participating in the PDAs at the beginning of the simulation. Each broker estimates its own energy requirement, and knows its own type. However, it does not know the types and requirements of the competing brokers. Every broker is allowed to submit an unlimited number of bids for each auction. After clearance, the clearing price and total cleared quantity of the auction is made public to all the brokers, while the last cleared bid or ask is not revealed. Additionally, each broker is privately notified about the cleared quantity and clearing price of any of its cleared bids/asks. The orderbook of the auction, which is the set of uncleared bids and asks without identity of the bidders, is also made public to all the brokers. If a broker fails to balance its retail demand portfolio after all the 24 auctions in the wholesale market, the balancing market automatically supplies the energy while charging the broker a

for its imbalance. The is comparatively higher than the wholesale market price, and is meant to penalize the broker for having an imbalance. For more details about the PowerTAC simulation, we refer the reader to the Power TAC 2018 Game Specification ketter20172018.

3 Related Work

Most bidding strategies for double auctions are designed for Continuous Double Auctions (CDAs) and would need to be modified for PDAs. Bidding strategies for PDAs, outside PowerTAC, are very limited. wah2016strategic (wah2016strategic) showed that in equilibrium, slow traders have higher welfare compared to fast traders in PDAs. As for bidding strategies for the PowerTAC wholesale market, AstonTAC kuate2013intelligent

uses Non-Homogeneous Hidden Markov Models (NHHMM) to predict energy demand and clearing price, which are then fed to an MDP to determine bid prices. TacTex

urieli2014tactex; urieli2016tactex; urieli2016mdp uses an MDP and dynamic programming based strategy derived from Tesauro’s bidding strategy to predict bid prices, which is the motivation for our MDP-based strategy. chowdhury2016predicting chowdhury2016predicting

predicts bid prices for the wholesale market PDAs using REPTree, Linear Regression and NN with weather data, with the former being is used in the SPOT

Chowdhury2017 broker. chowdhury2018bidding chowdhury2018bidding use a Monte Carlo Tree Search (MCTS) based strategy coupled with a REPTree based price predictor chowdhury2016predicting

and heuristics, to determine optimal bid prices. AgentUDE

ozdemir2015agentude uses an adaptive Q-learning based strategy in the wholesale market. None of these strategies are backed up by game theoretic analysis, where as our work is to build strategies derived from Nash Equilibrium.

4 Theoretical Approach and Proofs

In this section, we focus solely on the best response and Nash Equilibrium analysis of double auctions.

4.1 Nash Equilibrium analysis in single unit Double Auctions

Consider a single unit double auction, with the clearing price and payment rule given by ACPR. To find a generic Nash Equilibrium in this setting, we first try to simplify the double auction by restricting the number of buyers and sellers and their behavior. Upon doing so, we derive the following case-wise results.

4.1.1 One buyer and One Seller (OBOS)

Let’s assume that one buyer and one seller participate in the double auction, with their types as and respectively. We assume that both deploy scaling based strategies, i.e., a bid by a buyer is and an ask by the seller is where and are the scale factors by which the buyer and seller scale their true types while bidding, respectively. Motivated by the literature rothkopf1980equilibrium vincent1995bidding narahari2014game, we choose scale based bidding strategies for this Nash Equilibrium analysis, as compared to additive bidding strategies.

We assume and and this is common knwoledge. We also assume Equation (1), which states that the buyer’s bid (seller’s ask) at any point will be less (higher) than or equal to the highest (lowest) possible seller’s ask (buyer’s bid).

(1)

Thus, the utility of the buyer if its bid gets cleared, is denoted by the difference of true valuation and clearing price. Given the true types are picked over a distribution, the expected utility is computed as:

(2)

Now assuming that the buyer decides to fix its before even seeing its own type, then its utility is given by:

(3)

Now, differentiating w.r.t. and equating to 0 to find maxima:

(4)

Similarly, for the seller, the utility comes out to be:

(5)

Again, assuming that the seller decides to fix its before even seeing its own type, then its utility is given by:

(6)

Now, differentiating w.r.t and equating to 0 to find maxima

(7)

Next, simplifying the expressions for and by letting = and = , we get

(8)
(9)

Putting = = and = = in Equations (8) and (9), we get = and = . The above discussion is summarized as the following theorem.

Theorem 1.

For a single unit double auction with ACPR, with only one buyer and one seller, whose true types are drawn from a uniform distribution, if they deploy scaling based bidding strategies and which satisfy Equation (1) and fix their scaling factors and before seeing their true types, then = and = constitute a Nash Equilibrium.

4.1.2 Two Buyers and One Seller (TBOS)

Let’s assume that two buyers B1 and B2, and one seller participate in the double auction, with types , and respectively. We assume that all deploy scaling based strategies, and both buyers have the same scaling factor . Thus, a bid by buyer B1 is and by buyer B2 is , while a bid by the seller is . We also assume Equation (10), which states that the first buyer’s (seller’s) bid at any point will be less than or equal to the highest possible seller’s (buyer’s) bid.

(10)

First, we find the utility of the first buyer. We consider the following cases:

  1. and

    Let the utility in this case be denoted by .

    (11)
  2. and

    Let the utility in this case be denoted by .

    (12)

Now assuming that the first buyer decides to fix its before even seeing its own type, then we find the utility to be -

(13)

Now, differentiating w.r.t and equating to 0 to find maxima

(14)

Similarly, for the seller we find the utility. We again have 4 cases:

  1. and

    Let the utility in this case be denoted by .

    (15)
  2. and

    Let the utility in this case be denoted by . Since the two buyers are symmetric, the utility in this case comes to be same as in case 1.

    (16)
  3. and

    Let the utility in this case be denoted by .

    (17)
  4. and

    Let the utility in this case be denoted by . Since the two buyers are symmetric, the utility in this case comes to be same as in case 3.

    (18)

Now assuming that the seller decides to fix its before even seeing its own type, then we find the utility to be -

(19)

Now, differentiating w.r.t and equating to 0 to find maxima

(20)

From Equation (14), we have a bi-variate cubic equation in and , and from Equation (20), we have a bi-variate quadratic equation in and .

Assuming and (non-zero bids), and putting = = and = = in Equation (14), we get

(21)

Now, putting (from Equation (21)), = = and = = in Equation (20), we get

(22)

Since = (negative scaling factor), we ignore this solution.

Thus, putting = = and = = in Equation (14) and Equation (20), we get = and = .

The above discussion can be summarized as the following theorem.

Theorem 2.

For a single unit double auction with ACPR with two buyers and one seller, whose true types are drawn from a uniform distribution, if they deploy scaling based strategies , and , with buyers having the same scaling factor , which satisfy Equation (10) and fix their scaling factors and before seeing their true types, then and constitute a Nash Equilibrium.

As seen, with the increase in just one buyer, the complexity of the solution increases. It becomes increasingly difficult to extend and generalize the above results for a realistic market setting. Thus, moving forward, taking the PowerTAC wholesale market as testbed, we present a bidding strategy and experimentally show that it follows the theoretical results obtained in this section.

4.2 Best Response analysis in multi-unit Double Auctions with complete information

In practice, there are key differences between double auctions implemented in markets, and the theoretical results arrived above, stated as follows:

  1. Quantity may be involved in the trading market auctions, which is not considered above.

  2. The seller needs to use the same bidding strategy for one to achieve the above result, which may not the case.

So, considering a multi-unit double auction with , where bids are of the form . Let denote the quantity not cleared of the Last Cleared Ask if it is executed partially, and let denote the quantity not cleared of the Last Cleared Bid if it is executed partially.

Claim 1.

Upon clearance of an auction, either or , or both have to be zero.

Proof.

If the last bid partially clears, and , and if the last ask partially clears, and . If both clear fully, and . The last bid and last ask both can not clear partially, as, if they did, then more quantity can be cleared with last bid’s price higher than the last ask’s price. ∎

Next, we propose the best response if all the other bids are known to the bidder (i.e. complete information).

Proposition 1.

When a buyer (seller) has complete information about the auction, and it desires to procure (sell) entire energy it bids (asks) for, it’s a best response to bid as close as possible to the last clearing bid (ask).

Proof.

Let