The deployment of distributed energy stations (DESs) based on the internet built by the development of smart cities has been a hot topic because of its great potential to reduce the consumption of fossil fuels and curb greenhouse gas emission . Consider a smart community equipped with a DES, it is used to supply residents in this community with multiple energies, such as electricity and heat. DES existing in the community can reduce residents’ dependence on the centralized supply of energies, such as electricity from power grid and heat from heat station, thus save resources and reduce the cost of using energies. Moreover, it can sell surplus electricity and heat to the aggregators of power grid and heat station for making revenue. DESs can trade their surplus energies with aggregators that are responsible for collecting energies from their communities in a peer-to-peer (P2P) manner, thereby the multiple energies trading problem discussed in this paper is formulated.
Traditional P2P energies trading is performed on the centralized energy management platform, however, such a mechanism has many drawbacks. Traders often worry that their payment security and privacy protection when trading in an untrusted and opaque third centralized platform. This intermediary needs to verify and manage transactions between aggregators and DESs. If troubled by some damages such as single point of failure, it will lead to privacy leakage and transaction loss . Thus, it is urgent to create a secure energies trading system to guarantee trading among the distributed internet of energy can be executed effectively. It encourages the DESs to sell their energies to aggregators without worry, which promotes the rational use of energies.
Blockchain is a public and distributed database that is designed to store verified transactions among all valid participants without a trusted intermediary. Here, a new transaction is required to be validated by a group of authorized participants, and then it can be added into the blockchain in a permanent and tamper-resistant manner. It can be used to construct a secure and reliable energies trading system because of its decentralization, security, and anonymity  . Consider a smart city, it consists of a number of communities, each of which is equipped with a DES. There are two aggregators, electricity aggregator (EA) and heat aggregator (HA), trading with DESs in this city. The aggregators of different cities are interconnected to form a wide area network. Based on that, we propose a blockchain-based multiple energies trading (B-MET) system, where all aggregators are authorized participants required to store the blockchain and complete the consensus process. Thus this is a consortium blockchain as well, which is a little different from the classical public blockchain used in Bitcoin and Ethereum. Here, consortium blockchain is more convenient and flexible to achieve trading functions.
Based on such an architecture, we design a smart contract that ensures energies trading to be performed automatically when the trading conditions are satisfied. However, the proof-based consensus mechanism such as proof-of-work that is adopted by the most of blockchain applications is not suitable to our consortium blockchain in B-MET system because of its high latency, low throughput, and demanding computing power requirement. To finish the task of energies trading, it needs low latency and high throughput consensus mechanism. Thereby we design a new byzantine-based consensus mechanism (BCM) based on node’s credit, which reflects the performance of this node in the previous experience of participating in consensus. After each round of consensus, each node’s credit should be updated according to its voting result. If its voting is consistent with the result of consensus, its credit will be increased; otherwise will be decreased. Their credits affect directly their probabilities of being chosen as the leader and voting weight in the next round. This not only motivates participants to make the right decision, but also speeds up the consensus process.
In the aforementioned contract, there is an interaction between aggregator and DES before initiate a new energies trading, where the aggregator offers a unit price to purchase a kind of energy from DES, then DES decides the amount of energy they are willing to sell. In this paper, we take combined heat and power (CHP) system as an instance of DES, and aggregators are EA and HA. In a smart city, for each DES in this city, its utility consists of two parts: one is to serve the residents living in the community for satisfying their Daily consumption, and the other is sold to the aggregators for gaining revenues. For the aggregators in this city, their gains come from buying energies from DESs at a lower price and selling them at the retail price. To motivate the DESs to sell more energies, the aggregators should offer a higher price to them, but doing so raises costs and may result in lower overall profits. Since the multilevel decision-making processes between aggregators and DESs in a city, we formulate a novel multi-leader multi-follower (MLMF) Stackelberg game to model this bargain between them. Here the aggregators are leaders and DESs are followers. Their goals are to maximize their utilities or profits respectively. This MLMF Stackelberg game is analyzed and solved thoroughly in this paper, and we prove the Nash equilibrium (NE) among aggregators exists and is unique. Because the DESs are always able to respond aggregators with the optimal strategy according to their offered prices, the Stackelberg equilibrium (SE) exists and is unique as well. We propose a distributed algorithm that is guaranteed to reach the unique SE by limited information interactions. Finally, we conduct extensive numerical simulations to test the B-MET system, verify the correctness of our proposed utility functions and feasibility of our proposed algorithm.
The rest of this paper is organized as follows: Sec. II discusses the-state-of-art work. Sec. III introduces the architecture of B-MET system, describes CHP system, and defines utility functions. Sec. IV presents smart contract and byzantine-based blockchain. Sec. V introduces the Stackelberg game and discuess the solving process. Sec. VI conducts numerical simulations. Sec. VII is conclusion.
Ii Related Work
Distributed energy systems have been applied widely in many different forms, such as DES  and vehicle-to-grid  , to curb greenhouse gas and save cost. Integrating DESs into a smart grid  has attracted more and more researchers to participate in recently. This rouse the problems of energy management and energy trading problem. Cecati et al.  exploited DES to make the cost of power delivery minimized by use of an efficient smart grid management system. Georgilakis et al. 
summarized the optimally distributed generation placement problem systematically, classified and analyzed current and future research about it. Zhanget al.  considered microgrid as a local energy supplier for domestic buildings by utilizing DES, and studied optimal scheduling of energy consumption through mixed-integer programming. However, they only focused on electricity trading between grid and DESs, in this paper we consider multiple energies trading due to the diversity of energy forms.
In P2P energy trading, blockchain technology has been introduced to address transaction security issues. Kang et al.  put forward a localized P2P electricity trading pattern based on consortium blockchain among plug-in hybrid electric vehicles. Li et al.  proposed a P2P energy trading architecture based on consortium blockchain for the industrial internet of things relied on a credit-based payment scheme. zhou et al.  considered the scenario of vehicle-to-grid, and developed a secure energy trading mechanism based on consortium blockchain. Guo et al.  studied a blockchain-based energy management system that guarantees secure electricity trading between grid and DESs. However, they lose sight of low throughput and high latency in their proof-based consensus process, in this paper we try to address it by proposing a new byzantine-based consensus mechanism.
Stackelberg game is an effective tool to model the interactions in energies trading. Maharjan et al.  studied the demand response management by establishing a Stackelberg game between multiple utility companies and customers to maximized their utilities respectively. Bu et al.  proposed a four-stage Stackelberg game to consider a real-time pricing problem for the electricity retailer in the demand-side management. Yao et al. 
modeled the interactions between cloud server and mines by Stackelberg game, and solved it by multiagent reinforcement learning algorithm. Chenet al.  proposed a Stackelberg game-based framework to simulate the multiple resources allocation between cloud server and end users, and found an equilibrium solution by a backward induction process. However, most of these models have only one leader, in this paper the interactions between aggregators and DESs are MLMF, more complex and realistic.
Iii System Architecture
Consider a smart city, it consists of a number of disjoint smart communities, each of which is equipped with a distributed energy station (DES) responsible for supplying multiple energies, such as electricity and heat, to these residents living in this community. In this city, there are several aggregators, which represent different companies respectively, collecting different kinds of energies from all DESs appertained to this city. The architecture of blockchain-based multiple energies trading (B-MET) system is shown in Fig. 1. In the B-MET system, given a smart city , the entities in this smart city can be shown as follows:
1) Aggregators: There are two aggregators, electricity aggregator () and heat aggregator (), associated with this smart city . The (resp. ) is delegated by power grid (resp. heat station) as a monopoly of the energy market. They purchase electric energy (resp. heat energy) generated by DESs in those communities that belong to this smart city.
2) DESs: The city can be partitioned into an uncertain number of disjoint smart communities, denoted by set . In community , there is a distributed energy station supplying electricity and heat to the residents living in this community. Besides, is able to sell surplus electric energy (resp. heat energy) to the corresponding (resp. ) in order to make revenues.
3) Smart meters: It is a built-in component installed in each aggregator that monitors the energy flow transferred by each DES in this city in real-time, and decide whether the transaction has been accomplished.
Then, consider a larger ecosystem, such as a country, it is composed of a number of smart cities. This ecosystem can be denoted by . Here, each is a smart city in this ecosystem, and . For convenience, the notation can be considered equivalent to . Our B-MET system is established on such an ecosystem, in which all aggregators, including EAs and HAs, are interconnected each other to form a peer-to-peer (P2P) network called “blockchain network”, shown in the upper half of Fig. 1. In order to support secure energy trading between aggregators and DESs, we adopt consortium blockchain to construct our B-MET. In traditional blockchain, the consensus process is carried out by all participants. But the blockchain in the B-MET takes all aggregators in the ecosystem as authorized participants, and they are charged with storing the whole blockchain and performing the consensus process. Each aggregator manages and records those transactions between it and DESs in its city. The transactions are packaged into blocks and added into blockchain when the consensus among aggregators is reached, thus stored in all aggregators permanently.
Iii-a Combined Heat and Power System
Here, the aforementioned DES is implemented by the combined heat and power (CHP) system. The CHP system consumes natural gas to generate electricity and heat that serve its community or sell to the aggregators of its corresponding city, shown in Fig. 2. The gas is fed into the gas turbine (GT) which will generate electricity and emit high-temperature waste heat . The heat can be recovered by heat recovery system that can generate heat . Here, (resp. ) is used to supply electricity (resp. heat) to community, and (resp. ) is sold to EA (resp. HA).
Shown as Fig. 2, the total electricity generated by GT is . Measured in days, the units of quantities denoted by and are . The gas consumption per day can be defined as
where is the calorific value of natural gas, thereby the total energy generated by is definitely. The is the electric conversion of GT, percentage energy that transferred to electricity. Given a specific GT, its electric conversion can be considered as a constant. Besides, let be the thermal efficiency of heat recovery system, and be the thermal efficiency of heat component. We have and respectively.
As mentioned above, for a given CHP system, the electricity (resp. heat) generated by it can be divided into two parts: one is used to serve local residents, another is sold to EA (resp. HA). Thus, we define two dispatching factor for this CHP, where is the electric dispatching factor and is the heat dispatching factor. This DES needs to buy natural gas from the gas company. The company is for profit, thus it is valid to assume the gas company always supply enough gas that is able to meet the DES’s requirement. Given a smart city and a smart community , the energy relationships in the CHP system of has been obtained, that is
In this model, we assume all the CHP equipments in this ecosystem has the same efficiency parameters and . Each can determine the amount of electricity (resp. heat) that can be sold to (resp. ) by adjusting its dispatching factor (resp. ) automonously. For example, when is one, it means that will not sell any electricity to for making revenue.
Iii-B Utility Functions
Consider a smart city , the (resp. ) offers a unit price (resp. ) to collect surplus electricity (resp. heat) generated by , where the units of and are . For each , it is a risk-averse agent in the energy market. If chooses dispatching factor , , and consume natural gas , that is
where (resp. ) is the satisfaction function of community that provides electricity (resp. heat) to satisfy the usage of local residents in this community, and , , , and are defined from (2) to (5). Here, is the unit cost of natural gas.
From our simplified CHP model, we denote the cost of electricity (resp. heat) produced from by (resp. ). Then, the cost of electricity and heat can be quantified, that is and . Thus, we have . If the price (resp. ) offered by (resp. ) is less than the cost (resp. ), this will not sell any electricity (resp. heat) to them. It will reduce the gas intake such that only meet its local requirement. Like this, there is no energies trading between aggregators and DESs, and obviously, it is not what we want to see. Thus, it is reasonable to consider the prices offered by aggregators satisfy and . At this time, for each , it will produce electricity and heat as much as possible, because of the fact that it is always profitable to sell them to the aggregators. For maximizing its utility, each CHP system will run at full capacity. Here, for each , we define its maximum production capacity (maximum gas consumption) per day as . Therefore, the utility can be denoted by , because is considered as a constant.
After here, we denote and for convenience.
where (resp. ) is a non-negative satisfaction coefficient for electricity (resp. heat) in community , and (resp. ) is a non-negative adaption coefficient electricity (resp. heat) in this community as well. The adaption coefficients were proposed in  first, which aimed to control the variation range of the term , avoid it growing infinitely. Generally, we let (resp. ) when we choose (resp. ) by setting a valid adaption coefficient (resp. ) . Base on that, thereby we can formulate and as follows:
For aggregators in this city, power grid and heat station are the retailers for electricity and heat, however they do not have pricing power, because the retail prices of electricity and heat subject to government’s regulation. Hence, we define a retail price (resp. ) of electricity (resp. heat). As a selfish participant, it requires that and . From (6), if (resp. ) offered by (resp. ) is too low, each will respond with raising its dispatching factor (resp. ), and sell less energy to aggregators. If the aggregators offer a high price to purchase energy, their profitable spaces are reduced even if DESs are willing to sell more energies to them. Both of these cases will cause aggregators’ profit to be cut down. Therefore, it is important for aggregators to offer a optimal price such that not only encourage DESs to sell more energies, but also ensure sufficient profitability. If (resp. ) offers a price (resp. ), its profit function can be defined as
where (resp. ) is the profit function of the (resp. ) that collects electricity (resp. heat) from DESs in its city, and and are defined in (3) and (5).
Iv Byzantine-Based Blockchain
In this section, we will introduce a smart contract used to perform energies trading, and design a novel byzantine-based consensus mechanism based on the B-MET system.
Iv-a Smart Contract
A smart contract is a collection of programmable digital agreement that every participant commit to comply. Under our blockchain-based energies trading ecosystem, a transaction can only happen between aggregators and DESs in the same city. Thereby, consider a city , a smart contract can be decided together by its participants, which consist of an aggregator and a . We denote such a smart contract by . Between anonymous and untrusted entities in a city, the smart contract is able to execute credible transactions without third institutions. Then, the procedure of its smart contract is presented as follows:
1) System initialization: At the beginning, each needs to acquire a unique identification by registering in the designated institution authorized by government. It will be assigned with its public/private key pair and a . That is
where each account is associated with its wallet address and balance, . Then, for each aggregator in this city, it has those necessary information as well. But there is a credit value that represent the reputation of aggregator , thereby we have . Here, technologies of asymmetric encryption are usually adopted by current blockchain system for the sake of security, privacy, and data integrity. Given a massage encrypted by , we have , where the unforgeability and integrity is guaranteed.
2) Creation: An aggregator offers a price to buy energy from communities in its city, then responds it with the amount of energy that can be sold to . Like this, a new smart contract is generated by signing with their private key respectively. Then, this contract will be broadcasted to all authorized participants (aggregators) in the ecosystem . After reaching a consensus, this smart contract will be deployed and executed automatically. Each smart contract between aggregator and DES, , is associated with several variables, which include account information , offered price , amount of energy , expected transaction time , and timestamp . To guarantee this contract can be executed successfully, it needs to verify whether aggregator has sufficient balance such that and whether has enough production capacity to supply amount of corresponding energy on time.
3) Execution: The will be executed if current time after reaching a consensus among aggregators in blockchain network. From now on, it begins to trade energy and finish payment. The smart meter in aggregator verifies whether the amount of energy has been transported to the designated location. Then, fed this result from smart meter into the smart contract, if yes, it will execute the payment process automatically, that is
Here, we design a mechanism that the balance is permitted to be negative. At the moment of payment, the smart contract will complete payment as usual if the’s balance is not enough to pay. In this way, the ’s balance will become negative. Then, any contract that aggregator participants in will not be executed until its balance back to be positive.
Generally speaking, the energies trading between aggregators and DESs can be summarized as follows: In a smart city, a DES begins a smart contract with an aggregator by responding it according to its offered price. This contract needs to be verified by the consensus process in blockchain network. Then, it will execute the predefined procedure automatically once the trading conditions are met, which achieves the digital currency and energy exchange specified by contract between participants in a secure manner.
Iv-B Byzantine-based Consensus mechanism
As mentioned early, a consensus process is necessary to be performed so as to ensure the consistency of blockchain stored in every authorized node. Castro et al.  proposed a practical byzantine fault tolerance (PBFT) algorithm, which has been used in consortium blockchain system widely. Based on it and combined with the characteristics of energies trading, we design a new byzantine-based consensus mechanism (BCM). Our consensus process is performed among all aggregators in the ecosystem , and each of them has a local transaction pool (LTP) to store all transactions it receives. The consensus process is executed round by round, and the time interval of block generation is given by . There are three main stages, shown in Fig. 3, that is pre-prepare, prepare, and commit.
First, let us introduce the credit model, which will be used in leader election and to decide whether to reach a consensus. Let be the collection of consensus nodes. We have known that there is an attribute for each , where a larger implies node is more trustworthy. We denote by the credit of node after finishing the -th round consensus. Then, we can define according to the result of consensus in the -th round, where this result is whether to add the leader’s block into the blockchain. That is: (1) when is the leader, we have if its block is accepted to be added into the blockchain, else if its block is rejected; and (2) when is not the leader, we have if its decision is consistent with consensus result; else if it disagrees with the majority. We usually give and initialize the credit of each consensus node as . Consider entering the -th round consensus, the detailed process of BCM is represented as follows:
1) Leader election: The first step in this round is to select a leader from all consensus nodes. This leader election is based on node’s credit. Generally speaking, the better the credit value of a node, the more likely it is to be elected as the leader. Thus, the result of leader selection is unpredictable. For a node , the probability that it is elected as the leader of the -th round consensus is ,
where represents the leader of the -th round consensus. Obviously, there is no chance to select a node whose credit is zero as the leader.
2) Broadcast: Each aggregator in broadcasts all transactions which happen in current and co-signed with a DES in its city to the blockchain network. All the consensus nodes will verify whether their received transactions are valid. Those valid transactions will be stored in their LTP, and invalid transactions will be discarded.
3) Pre-prepare: After all non-leader consensus nodes in have completed above verification process for received transactions, the leader will package those selected valid transactions in its LTP into a block . Then, the leader signs this block and broadcasts pre-prepare message - to the blockchain network.
4) Prepare: For each non-leader node , it will check the identity of leader and verify the pre-prepare message from the leader. The block verification needs to confirm the pointer to the previous block, mercle root is correct and compare the transactions in with the corresponding transactions in its LTP. If node believes is valid, it broadcasts this prepare message to the blockchain network. All consensus nodes must make decisions in this step, whether to agree or disagree with adding block into the blockchain. Then for each node , it gathers all prepare massages from other consensus node, checks their identities and counts the weighted sum of received prepare messages. Let be the set of nodes from which node receives prepare messages, including itself. If satisfying the following inequality
where , we say node will accept block and broadcast commit message to the blockchain network.
5) Commit: After sending their commit messages, they should waiting commit messages from other consensus node. For each node , its consensus process is completed until it recieve sufficient commit messages such that , where is the set of nodes from which node receives commit message, including itself.
5) Add a block and update credits: If a consensus node accepts the new block , it will be appended into the blockchain in a linear and chronological order, which includes a pointer to the previous block. Any failure occurs in these three stages will terminate the consensus of current round (do not add the new block). Besides, before finishing this round, we need to update the credits of all the consensus nodes according to the credit model. In next round, the consensus process will perform based on their new credits.
Failures that terminate the current round and do not add a new block mainly include the following reasons: (1) the leader sends invalid block or do not send its packaged block before the deadline; and (2) too many malicious nodes do not breadcast prepare messages even though this block is valid. Shown as node in Fig. 3, it is a faulty node. The credit of those nodes that make mistakes in this consensus process will be reduced. Let be the number of malicious nodes. According to , supposing , the faults can be tolerated by the consensus system with nodes. In our BCM, each consensus node’s credit is initialized as a constant, thereby good nodes can make sure that (13) is satisfied. As the consensus process performs more and more times, the good nodes’ credit increase but malicious nodes’ credit decrease gradually. Therefore, the credits in system will be more accumulative in good nodes. According to (13), the number of prepare message and commit message required to reach a consensus declines, which helps reduce latency and improve throughput. In summary, secure and traceable energies trading and digital currency exchange can be guaranteed by our proposed B-MET system.
V Multiple Energies Trading: A Stackelberg Approach
A non-cooperative Stackelberg game generally refers to the multilevel decision making processes of a number of independent decision-makers in response to the decision taken by the leading player of the game . In this section, we put forward a multi-leader multi-follower (MLMF) Stackelberg game to model the interactions in above smart contract between aggregators and DESs. Consider a smart city , the MLMF Stackelberg game can be defined as
where the components are shown as follows:
1) Players set : The aggregators and act as leaders, and offer a price respectively to the DESs. Then, act as followers, and decide on the amount of electricity and heat they want to sell respectively according to the offered prices.
2) Strategy spaces and : Let be the strategy space of two aggregators, where we say is a feasible strategy of and . Then, let be the strategy space of all DESs in this city, and we have is a feasible strategy of DESs.
3) Utility functions and : Each player in this game aims to maximize its utility or profit, which reflects the quality of strategy that this player chooses. is the profits of aggregators, defined in (9) and (10); and are the utilities of DESs in , defined in (6).
V-a DESs (Followers) Side Analysis
Given a price strategy offered by two aggregators in city , each decides the amount of electricity (resp. heat ) that sold to the (resp. ) by adjusting its dispatching factor (resp. ). Thus, each aims to choose its optimal dispatching factors according to by solving the following optimization problem (), that is
where is the minimum amount of energy that is required to maintain the basic life for those residents living in this community. The objective function, shown in (6), is strictly concave and continuously differentiable, this is a convex optimization problem, which will be proved later. Thus, the stationary solution is unique and optimal.
To ensure reasonableness, the should be in a valid range, thus we have . It means that this minimum requirement is larger than the production capacity of electricity or heat separately, which implies that and are impossible to approach zero definitely. Hence, the restrictions on (16) can be converted equivalently to constraint (17), that is
Then, its first-order derivatives is
Let and , we have
Here, we need to note that the setting of parameter (resp. ) must be in a valid range such that (resp. ) for any offered price (resp. ). Or else, this utility function is monotone, and it is meaningless to adjust its dispatching factors. Base on that, thereby we can restrict and as follows:
where it assume (resp. ), or else no such (resp. ) can keep (resp. ) satisfied for any offered prices.
Sequentially, we use Lagrange’s multipliers , and for constraint (17), thereby the , shown as (15) and (17), can be converted to the following form , that is
where we denote . Then, Based on (22), the complementary slackness conditions (KKT conditions) of are demonstrated as follows:
The optimal solutions of , shown as (15) and (17), can take one of the following four cases, that is
1) Case 1: For and , we have . Look at (25), if , substitute it into (23) and (24), we can get a solution according to (20). Then, we need to check whether constraint (17) can be satisfied. If yes, the optimal solution is . If no, it means and . At this time, by solving (23) and (24), we have
Substitute (29) and (30) into (25),
where , , and . By solving (31), we have two solutions, they are
Here, it is easy to verify and based on (9), (21), and , thus it is possible that . If , there is no real solution; else we need to check whether the defined on (32) satisfies . If , substitute (32) into (29) and (30), we obtain a solution . If it is feasible, namely , the optimal solution can be determined by .
2) Case 2: For and , we have . Look at (25), if , substitute it into (24), we can get a solution according to (20). Then, we need to check whether constraint (17) can be satisfied and . If yes, the optimal solution . If no, it means and . According to (24), we have which is shown as (30). Substitute (30) into (25),
By solving (33), we have
If and , substitute (34) into (30), we obtain a solution . If we have , the optimal solution can be determined by .
3) Case 3: For and , we have . Look at (25), if , substitute it into (23), we can get a solution according to (18). Then, we need to check whether constraint (17) can be satisfied and . If yes, the optimal solution is . If no, it means and . According to (23), we have which is shown as (29). Substitute (29) into (25),
By solving (35), we have
If and , substitute (36) into (29), we obtain a solution . If we have , the optimal solution can be determined by .
4) Case 4: For and , we have because we have assumed before. Substitute it into (23) and (24), we have
By solving (37) and (38), we have
According to (9) (21), the maximum value of can be obtained when giving . Substitute it into (39), we have because of . By using the same way, we have