The rapid advance of mobile edge computing (MEC) has been the last mile of enabling a shared, low-latency computational environment for multi-vendor mobile edge applications. MEC performs computing offloading, data storage, caching and processing, request distribution and service delivery from the mobile edge to end users . Applications with low latency tolerance, such as augmented reality (AR), video streaming, and online gaming, can deploy their services on the edge hosts at a cost, to achieve lower latency and better user experience .
As the market gets mature, there will be multiple 5G service providers (SPs) provisioning MEC services to cover the same area: bigger wholesale players will invest in infrastructure to actually build mobile edge base stations, while there will also be mobile virtual network operators (MVNOs) renting resources from the former. These SPs can collaborate with each other in several ways for better utilization of the resources at the edge: virtual SPs have to place mobile edge (ME) applications on one of the rented edge hosts, preferably with lower cost, regardless of SPs. On the other hand, MEC base stations from different SPs can share resources with each other to process bursting requests.
For encouraging SPs to enroll their eligible MEC base stations and hosts in resource sharing, it is common to give incentives to SPs for contributing their resources of the hosts for hosting edge applications. Following the changing demand of end users, certain types of edge applications need to be deployed on, migrated to, or removed from an edge host, in order to meet the service requirement. By deploying the edge applications at the right places, the edge application provider will save costs, while providing high-quality service with low latency to the end users. Meanwhile, the edge host will collect incentives for its resources effectively used.
Clearly, the edge computing framework needs a placement service to dynamically check the user needs and the available edge hosts, and determine the placement or removal of edge applications. In datacenters, virtual machine (VM) placement has been well investigated, mainly with the focus of more efficient resource utilization and lower operational expense (OPEX). However, the collaboration of multiple SPs and mobile edge applications vendors are posing new challenges for ME application placement from the following aspects:
A placement model has to make transparent and consistent selections of the best host for each request for edge computing resources. Moreover, the model has to take into consideration that a mobile edge application may require multiple services chained together at the edge.
A trusted party is required to determine the best place for application deployment. When an edge application is deployed on a mobile edge host, the application vendor needs to pay for the usage of the host. The placement algorithm has to avoid affiliation to either SP to ensure a neutral decision is made strictly according to the resource and the cost. It may create conflicts of interest to put any SP involved into the position of making placement decisions: placing mobile edge applications onto the SP’s own hosts would bring revenue for renting their resources.
The application placement service needs to be steadily available. Both the mobile edge hosting service providers and the mobile edge application providers can constantly change. The placement service provider must remain in service regardless of the joining or quitting of vendors.
The challenges above urge a comprehensive solution uniting all SPs and their edge hosts without bias. In this paper, we present an architecture combined with its algorithm, namely EdgeChain, to create a decentralized placement service for mobile edge application that does not require trust to any party, i.e., trustless placement service. Compared with current placement solutions, EdgeChain has the following contributions:
A cost model is presented as a stochastic programming problem, factoring in the pricing of edge hosts, latency, and service chaining.
We develop a heuristic placement algorithm based on the proposed cost model with the consideration of efficiency for running by the blockchain.
We introduce blockchain technologies to the MEC resource orchestration framework with two considerations: the first is to store the global resource availability, allocation, and consumption information that helps our algorithm make optimized decisions based on the global resource information. The second consideration is to have a decentralized public ledge for ensuring the neutrality of the placement decisions.
The EdgeChain framework is presented to run our algorithm for making placement decisions. In our design, SPs and mobile edge application vendors participate in the maintenance of the blockchain. An EdgeChain client is embedded in the network function virtualization (NFV) framework to determine the placement based on the existing information on the blockchain. To our best knowledge, this is the first work that leverages blockchain to coordinate SPs for MEC application placement.
We divide the contents into the following sections. The related work is illustrated in Section II. Section III formulates the problem. Section IV proposes the heuristic EdgeChain placement algorithm based on the problem formulation. Then the simulation results are shown in Section VI. Section VII concludes the paper.
Ii Related Work
The research directions in network service chaining (NSC) were discussed in . For security considerations, the authors highlighted the difficulty of bringing short-lived network services to targeted users in a single subscriber network by using the current security schemes. The potential security problems in SFC were stated in RFC7498 , including service overlay security, trusted classification policy, and secure SFC encapsulation. We investigated a placement problem in MEC with the consideration of application availability in .
Xiong et al. proposed a pricing strategy for offloading the blockchain’s resource-consuming proof-of-work tasks to edge computing nodes . A two-stage Stackelberg game model was presented with both the edge computing service provider and the miners involved. A hierarchical distributed control system was built using Hyperledger Fabric blockchain . The hosting locations of cloud and fog of blockchain were compared in  for IoT networks with the conclusion that fog nodes were better as network latency was the dominant factor.
Nakamoto introduced the concept of blockchain and implemented Bitcoin , a decentralized cryptocurrency that first resolved the double spending problem. Blockchains are based on Merkle trees  to efficiently allow multiple documents to be saved together in a block. As a decentralized public ledger, blockchains can serve beyond cryptocurrencies. Ethereum  used blockchain to store smart contracts that support building virtually any decentralized application.
|Users||Subscribers of applications and services over 5G networks with MEC enabled.|
|MECSPs||MEC service providers, who deliver MEC hosting services that can run MEApps at the network edge, close to end users. Examples include telecommunication companies like Rogers and Telus in Canada.|
|MEAVs||Mobile edge application vendors, who provide MEApps and services to end users. For instance, a company selling AR services.|
|MEApps||MEApps stand for mobile edge applications provided by MEAVs.|
|MEHosts||Servers that belong to different MECSPs to provide hosting service of MEApps.|
|HostLinks||Network links between hosts, regardless of which MECSP they belong to.|
|AppLinks||When MEApps are chained together, virtual links will be established for data transmissions traveling through the chain.|
|is a service chain. is the set of all MEApps in . is the set of all AppLinks in . A MEApp in is denoted by , and an AppLink between two MEApps in is denoted by .|
|is the set of all MEHosts. is the set of all HostLinks. A MEHost is denoted by , and a link between two MEHosts is denoted by . is the set of all MEApps placed on .|
|is an end user. is a MECSP. is the cost of deploying .|
|is the cost of deploying . is the cost of the AppLink between on and on .|
is the total number of users requesting .
is a random variable denoting the percentage of the users of MECSP. is an edge host of .
|is the unit price of serving ’s own subscribers. is the extra charge for serving users of other MECSPs.|
|CPU and memory requirement of the MEApp .|
|is the total bandwidth capacity of HostLink . is the unit price of the bandwidth of .|
|is the total bandwidth used by MEApps deployed on and .|
|Bandwidth used between MEHosts and .|
|CPU and memory capacity of the MEHost .|
|is the latency incurred on HostLink . is the latency of the service chain . is the max latency allowed by .|
Iii Problem Formulation
We first list all parties involved in a MEC placement scenario in Table I. The problem is formulated from a MEAV’s point of view: MEApps are direct consumers of the computing resources in the MEC environment, because a MEAV needs to pay MECSPs for hosting its applications in order to serve their users and meet the latency requirement. Each MEApp is equivalent to a virtual machine (VM) deployed on a MEHost. MEApps provided by different MEAV can be combined as a service chain to provide comprehensive services. A service chain may span multiple MEAVs. In this case, revenues generated by the service chain can be distributed according to the usage of each MEApp on the service chain. For instance, a full-fledged AR service can load real-time navigation information from an online map application, while it can also load promotions of a shopping mall nearby from the mall’s application. The navigation data is collected by the online map application, and the shopping mall application gets paid if the user ”clicks” the links of the promotions.
The notations used in formulating the problem is shown in Table II. Define a chained service as a forwarding graph  , where is the set of all MEApps contributing to the service, and is the set of all AppLinks connecting applications together. A MEApp is denoted by , and an AppLink between two MEApps is denoted by .
The chained service is deployed on a graph of connected MEHosts , where is the set of all MEHosts owned by various MECSPs and is the set of all HostLinks. A MEHost is denoted by , and a HostLink between two MEHosts is denoted by . The HostLinks can be either physical or virtual links with fixed capacities and latencies.
Suppose in a certain service area, there are users from various MECSPs requesting the same chained service from a MEAV. We use to denote a MECSP and for a MEHost that belongs to . Define an assigning function , whose value is if VM is assigned to Host , 0 otherwise.
Define a binary indicator of an AppLink between two chained MEApps in , denoted by , such that
Also, we use to represent the HostLink between and . The cost of deploying is the sum of the cost of deploying each MEApp of the service and the cost of the traffic between each two adjacent MEApps in the service chain. It can be shown by
where represents the cost of deploying and is for the cost of a MEApp deployed on a MEHost . We assume that the pricing scheme for the same MECSP is the same across all of its hosts. For a MEHost , define its basic unit resource price, which is the unit price of serving its own subscribers, as . When is serving users of other MECSPs, it charges a premium of for its unit resource, as the return for doing courtesy for its partners. Therefore, the shared unit resource price of can be represented by . Define and to be the capacity of vCPU and memory provided by . Define and as the vCPU and memory consumed by . Define to be the random variable for percentage of the users using the service chain via networks of the MECSP . Depending on the numbers of active users for each MECSP, the total cost for the MEAV to place its MEApp onto a host of is the cost incurred by users of plus the cost by users of other MECSPs:
When a request from a user for a service chain arrives, the blockchain would know the MECSP from which the user subscribes. For the same placement decision, the value can significantly differ over changing distribution of users. An example can be two MECSPs and , each with one host and . If all users are subscribers of and all MEApps are placed on MEHosts of , then the cost payable by the MEAVs would be lower than if all users were subscribers of .
Iii-a HostLink Unit Price
When it comes to the cost modeling of a link between two MEHosts, link availability is an important part for the service consistency of the MEApps. If a heavily used HostLink is down, consequences can be catastrophic: even if all individual MEApps are running, the traffic would not be able to flow through between one or more pairs of MEApps and the service chain would not be functional. For each HostLink , there can be one or more AppLinks sharing its bandwidth. HostLink outages require migrating the MEApps if they cannot be fixed in time. Therefore, HostLink availability has significant influence on possible MEApp migrations and the potential costs incurred.
The link unit price of a HostLink , denoted by , is then defined to describe how much to use the HostLink . The following two parameters will determine .
The first parameter is as defined in Eqn. (2). The more AppLinks a HostLink carries, the more vital and expensive it becomes. The reason behind this ranking parameter is the potential consequence of migration: failure of a HostLink used by many VMs would lead to massive migration of all MEApps connected by that HostLink, which would be more disruptive to the service chain.
The other parameter is the total bandwidth consumed by traffic between MEApps on the two hosts. It is selected because larger bandwidth usages would cause challenges at the time of migration: it can be hard to find another link with enough capacity.
Combining the two parameters, we define the unit price of a HostLink , as the factor of the number of AppLinks between two hosts times the factor of traffic flowing through these links:
where is the maximum number of virtual links possible on . Therefore, . The value of will rise to mark up a link’s importance given it is either occupied by more pairs of VMs, or there is more traffic assigned to , or both. The cost of any two MEApps is then the sum of the cost serving users that belong to the MECSPs owning and and the cost serving other users timed by the price factor :
Iii-B HostLink latency
Define the latency of the link to be . For a service chain , the total latency is then
In the equation above, is a constant depending on the particular . If , then we consider the latency to be 0, since no actual HostLink is used for data transmission between the two MEApps. Define the maximum latency allowed for the service chain is . Then there must be to meet the latency requirement.
Iii-C Stochastic Programming Formulation
The problem is formulated as a stochastic programming optimization. Define as the set of all MEApps deployed on the MEHost . The objective is to minimize the total cost of the service chain to provide service with the lowest cost to the end user. As discussed in Section III, the optimization is to minimize the costs on MEHosts and HostLinks for all MEApps of .
Function (9) is the objective function. It minimizes the cost of all MEApps and AppLinks by using less hosts, while not exhausting them.
Constraint (10) is the HostLink bandwidth capacity bounds between each two hosts. Traffic transmitted between any two hosts and must not exceed the corresponding bandwidth capacity .
Constraint (13) is the latency requirement of the service chain to ensure that the total latency of must not exceed the maximum latency allowed .
Iv The EdgeChain Placement Algorithm
The formulation presented in the previous section is a stochastic programming problem. Problems of this type been proved to be NP-hard . It may not be computationally feasible when attempting to solve it in large scale. To apply our model to real-world scenarios, we design a heuristic algorithm called EdgeChain to achieve suboptimal results by applying a hybrid strategy of best-fit and first-fit decreasing algorithm. The pseudo code of the algorithm is shown in Algorithm 1.
Iv-a Processing Order and selection of MEHosts
The EdgePlace algorithm runs on each mining node based on the Ethereum platform. The algorithm retrieves its input information from the blockchain, as all transactions and updates are recorded on the blockchain. The EdgePlace algorithm will select the MEHosts following the steps below.
Sort all MEHosts by the percentage of users of the service chain. For each MEApp on the service chain, consider which MECSP has most users using it. Then MEHosts with the same MECSP will have higher ranks to deploy this MEApp. Since all MEHosts of the same MECSP have the same unit resource cost, the MEApp can be placed on any of the MEHosts that belongs to the best MECSP, to avoid the situation that too many MEApps are concentrated on one MEHost.
Iv-A2 Last-hop MEApp
For MEHosts given higher priority in the previous step, sort by the locations of last-hop MEApps. MEHosts hosting the previous-hop MEApps will be considered first. This step is to reduce the traffic cost between different MECSPs.
For MEHosts given higher priority in the previous step, sort by the latency of the HostLinks to the previous MEApps in the service chain. MEHosts with lower latency will be considered first.
After the list of candidate MEHosts are sorted according to the steps above, the algorithm iterates the list and pick the first valid MEHost that has enough resources to place the MEApp, as well as meeting the latency requirement of the service chain.
V EdgeChain Design and Implementation
In this section, we introduce the design and implementation of EdgeChain, a blockchain-based system that integrates with the existing MEC architecture for MECSPs and the scheduler of MEAV. There are mainly two reasons the blockchain is used in the system:
The blockchain acts as a public ledger that stores all useful information and transactions made during the placement process. Exposure of the information would help the placement algorithm make optimized decisions considering the global resource demand and allocation. The blockchain enables such centralized resource information, in a decentralized implementation.
As a public ledger applying proof-of-work verifications, the blockchain makes it nearly impossible to tamper the history stored in the blockchain. The EdgeChain algorithm will be downloaded by all mining nodes and they will execute the same algorithm with the same input. The placement result will only be accepted by the system if majority of the mining nodes reach agreement on the output. This will ensure the neutrality of the placement decisions.
The system takes requests to place MEApps from MEAVs, and the placement algorithm runs as the smart contract on the blockchain to select the best MEHost from all candidates. The NFV orchestrator of the related MECSP receives and enforces the placement decision, while posting the transaction onto the blockchain for recording. While this paper is written, the blockchain is implemented based on VeChain , an enterprise-level blockchain-as-a-service framework derived from Ethereum.
V-a Data Entities
As Fig. 3 shows, there exist 6 types of data entities on the blockchain and they are related to each other to represent the status of MEHosts and placement decision of running MEApps. The descriptions of these data entities are illustrated below. Each data entity record has a unique Ethereum address for other to locate it on the blockchain. All types of data entities can be created, updated and deleted, while the blockchain will keep the audit trail of every change.
When a MECSP record is registered to EdgeChain, a record of this MECSP is added with the Ethereum addresses pointing to the records of all its eligible MEHosts and HostLinks. A MECSP record is updated whenever there is change to any MEHost or HostLink.
A MEHost record registers under an existing MECSP to the blockchain. In a record, the vCPU and memory capabilities can be found, along with the Ethereum addresses pointing to the records of all MEApps placed onto it.
Similar to MEHosts, a HostLink is under a registered MECSP, which contains the two MEHosts it connects, and the bandwidth of the HostLink.
A service chain is registered by a user to the blockchain to reflect the resource consumption of a chained service, including that from MEApps and the corresponding AppLinks. The service chain can have MEApps from multiple MEAVs.
A MEAV will submit a record of a MEApp whenever it needs to spin up one. A record stores the vCPU, memory usage of the MEApp.
AppLinks describe chained relationship between two MEApps. The source and destination MEApps are stored in an AppLink record, as well as network bandwidth requirement of this link.
V-B EdgeChain Work Flow
A typical EdgeChain work flow can be demonstrated by Fig. 4, where there are three parties participating in the entire process: MECSPs, MEAVs, and mining nodes. We use circled numbers and alphabets to define the work flow in sequence.
A user requests a service chain from the blockchain. Such requests will be sent to the blockchain every time a user requests a service chain.
The request for the service chain is recorded. When the request is synced to the mining nodes, it will be broken into requests for MEApps. The mining nodes will run the logic to break down the service chain creation request. Then the requests for MEApps are propagated to all corresponding MEAVs.
Based on its user demand, the MEApp Scheduler decides to create a new instance of MEApp and pass the request to the Ethereum client of the MEAV.
The Ethereum client running the EdgeChain service sends the request to the blockchain, creating records for the request of placing a new MEApp.
The request of creating a new MEApp arrives at a MECSP through its Ethereum client.
For every MECSP, the Ethereum client requests the NFV Orchestrator (NFVO) to call the EdgeChain placement algorithm downloaded to the resource manager for the decision of the placement. This will ensure that the placement algorithm be executed by different parties for verifying the results. The placement result returned by the next step will only be accepted if majority of the parties return the same placement result.
The NFVO calls the EdgeChain placement algorithm for the placement decision. Note that the decision can be a hash representing any MEHost within the entire MEC network. If the result points to a MEHost which does not belong to the current MECSP, then no actual placement will be done. Instead, only the result along with the algorithm’s hash will be returned to the Ethereum client for verification.
If the result points to a MEHost of the current MECSP, then the NFVO will sends the request to place the MEApp to the VNF Manager (VNFM). Also, a transaction shown in Fig. 5 will be posted to the blockchain to record that placement actually occurs.
The VNFM sends the request to the NFV Infrastructure (NFVI) deploy the MEAPP onto the target MEHost.
The mining nodes periodically perform the mining process to verify the blockchain, as well as earning Ethers for requesting placement services. Meanwhile, the resource manager periodically synchronizes with the NFVI for the up-to-date resource usage and availability, and then posts the updated information to the blockchain.
Vi Numerical Results
In this section, we illustrate the numerical results of the MEC placement cost changes based on varying mobile edge application user cases using CloudSim . To clearly demonstrate the focused trends, the following assumptions are made to simplify the modeling of the problem without losing generality. We first discuss the placement results output by the EdgeChain algorithm for the same service chain on the same set of MEHosts.
The unit costs of the CPU and memory of all hosts for the same MECSP are the same.
Costs of network bandwidth for all links follow the same unit price.
One mobile edge application includes the same type of VMs with the same CPU, memory and network bandwidth requirements.
A request from the user will be processed by one VM, while the VM may communicate with other VMs to exchange information.
With the assumptions above, we choose parameters for our placement model to evaluate the performance and the facts under different circumstances. First, we choose a MEC service scenario of 3 MECSPs , , and , each with 3 MEHosts, where , , belong to , , , belong to , and , , belong to .
Three identical requested service chain, each with 5 MEApps is to be placed. The MEApps of each service chain are denoted by , , , , and . The service chain starts from and ends at : . We assume that all MEApps have the same CPU, memory and bandwidth requirements, which are shown in Table III, along with other parameters.
|10000 Mbps||30 Mbps|
|15 ms||50 ms|
Vi-B Placement trends with changing unit resource premium
The placement decision changes by the increase of under different user distributions are shown in Fig. 6, where , the unit resource premium payable to the MECSP for hosting MEApps for others, increases from 0.1 to 0.6. For comparison, in Fig. 6(a), most users are from . There is and . Meanwhile, in Fig. 6(b), most users subscribe services from as and .
From the results of the two scenarios, we learn that the MEHosts with lower combination of unit resource base price () and unit resource premiums () will be selected first. The MEHosts of the MECSP will have more weight upon consideration if there are more users from that MECSP.
Vi-C Placement trends with changing user distribution
To further demonstrate the impact from the distribution of the users, we simulate various scenarios with different percentages of users for and , while there is no user for . Users of increase from 0% to 100%, while those of decrease from 100% to 0%.
The results have shown the trends of MEApps migrating to MEHosts owned by the MECSP that has more active users to avoid premiums charged by other MECSPs. However, resource sharing still takes place ( hosting MEApps for and ) when needed for better latency results and service quality.
In this paper, we have presented the architecture and the algorithms for mobile edge applications placement for multiple mobile edge computing service providers, leveraging the blockchain-based system called EdgeChain. Future work will be considering multiple service chains initiated by multiple users, to achieve lower overall costs for the entire system.
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,”IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, Oct 2016.
-  Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young, “Mobile edge computing—a key technology towards 5g,” ETSI White Paper, vol. 11, 2015.
-  VeChain, https://www.vechain.com/, [Online; accessed Jan-10-2018].
-  G. Wood, “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum Project Yellow Paper, vol. 151, 2014.
-  W. John, K. Pentikousis, G. Agapiou, E. Jacob, M. Kind, A. Manzalini, F. Risso, D. Staessens, R. Steinert, and C. Meirosu, “Research directions in network service chaining,” in 2013 IEEE SDN for Future Networks and Services (SDN4FNS), Nov 2013, pp. 1–7.
-  P. Quinn and T. Nadeau, “Problem statement for service function chaining,” 2015.
-  H. Zhu and C. Huang, “Availability-aware mobile edge application placement in 5g networks,” in IEEE Globecom’17, Dec 2017.
-  Z. Xiong, S. Feng, D. Niyato, P. Wang, and Z. Han, “Edge computing resource management and pricing for mobile blockchain,” arXiv preprint arXiv:1710.01567, 2017.
-  A. Stanciu, “Blockchain based distributed control system for edge computing,” in 2017 21st International Conference on Control Systems and Computer Science (CSCS), May 2017, pp. 667–671.
-  M. Samaniego and R. Deters, “Blockchain as a service for iot,” in 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Dec 2016, pp. 433–436.
-  S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” 2008.
-  D. Bayer, S. Haber, and W. S. Stornetta, Improving the Efficiency and Reliability of Digital Time-Stamping. New York, NY: Springer New York, 1993, pp. 329–334.
-  G. Brown, “Service chaining in carrier networks,” 2015.
A. A. Gaivoronski, A. Lisser, R. Lopez, and H. Xu, “Knapsack problem with probability constraints,”Journal of Global Optimization, vol. 49, no. 3, pp. 397–413, 2011.
-  R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. De Rose, and R. Buyya, “Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms,” Software: Practice and experience, vol. 41, no. 1, pp. 23–50, 2011.