Mobile edge computing (MEC) that provides cloud services within the vicinity of mobile device (MD) via the radio access network can effectively enhance the computing power of MD and reduce latency-. Computation offloading, which leverages powerful MEC servers to augment the computing capability of less powerful MDs, is a key computing paradigm used in MEC. In , computation offloading strategy optimization with multiple heterogeneous servers in MEC is investigated. Three multi-variable optimization problems are studied, and corresponding efficient numerical algorithms are proposed. In 
, the computation offloading problem is formulated as graph cut problem, and a solution based on spectral clustering computation is developed. In, a lightweight and efficient user level online offloading framework for MEC, is presented. Real experiments with Android systems and simulations using large-scale data from a cellular network provider are preformed to prove the efficiency of the proposed framework. In , the multi-user computation offloading problem for mobile-edge cloud computing is studied under a multi-channel wireless interference environment. A multi-user computation offloading game is formulated, and a distributed Nash equilibrium (NE) achieving computation offloading algorithm is designed.
To prolong the lifetime of the battery, energy harvesting technology being capable of converting the energy from the environment (e.g., solar, ambient radio-frequency (RF) signals) into electrical energy has been widely utilized in wireless communications. The energy management of the stored harvested energy at the battery plays a centric rule in energy harvested aided wireless networks. In , energy efficient resource allocation in a time division multiple access (TDMA) based wireless energy harvesting sensor network is studied. The closed form expression for the optimization problem is obtained, and thereafter solved by utilizing Karush-Kuhn-Tucker (KKT) conditions. In , hybrid energy harvesting communication systems are modeled based on their probabilistic natures. An integrated Markov energy model is designed. In 
, beamforming implementation in energy harvesting wireless networks is surveyed. Different beamforming approaches in each network topology are classified according to its design objective.
Over the past years, massive multi-player online game, virtual reality (VR), and artificial intelligence (AI)-based applications (e.g., augmented reality and face detection) have emerged on smart MDs. High computation capacity requirement, low latency, heavy energy consumption, and the stringent equipment-size constraint have brought great challenges on smart MDs, especially for the CPU and battery. Combing the virtues of the two aforementioned technologies, energy harvesting MEC could augment the MD to handle the computation-intensive, latency-sensitive and energy-hungry applications. In, a green MEC system with energy harvesting devices has been considered, and an effective Lyapunov optimization-based dynamic computation offloading algorithm is developed. The proposed algorithm jointly decides the offloading decision, the CPU-cycle frequencies for mobile execution, and the transmit power for computation offloading. In 
, an efficient reinforcement learning-based resource management algorithm is designed for energy harvesting MEC. The proposed algorithm learns on-the-fly the optimal policy of dynamic workload offloading to the centralized cloud and edge server provisioning to minimize the long-term system cost (including both service delay and operational cost). In
, the computation offloading process in energy harvesting MEC is modeled as a Markov decision process (MDP) with no prior statistic information. Thereafter, reinforcement learning algorithms are utilized to derive the optimal offloading policy. In, the multi-user multi-task computation offloading problem for green MEC is investigated. Lyaponuv optimization approach is utilized to determine the energy harvesting policy and the task offloading schedule. Centralized and distributed greedy maximal scheduling algorithms are proposed. In , the energy harvesting edge computing is investigated, and an effective and efficient MDP-based energy harvesting and data transmission scheduling strategy is designed. In , computation offloading is discussed in an edge-computing system consisting of energy harvesting MDs and a dispatcher. Online rewards-optimal auction is developed to maximize the long-term sum-of-rewards for processing offloaded tasks. In , computation efficiency maximization problems are investigated in wireless-powered MEC networks. Partial and binary computation offloading modes for TDMA and non-orthogonal multiple access (NOMA) are respectively considered. In , deep reinforcement learning-based online offloading framework to maximize the weighted sum computation rate in wireless powered MEC networks with binary computation offloading is proposed. The proposed algorithm optimally adapts wireless resource allocations and task offloading decisions to the time-varying wireless channel conditions.
In this paper, we investigate the computation offloading in multiple heterogeneous MEC networks with energy harvesting. There are multiple MDs and multiple MEC serves (MEC-ss) from different MEC networks. Besides tackling locally, each energy harvesting aided MD could partially or fully offload its computation tasks to MEC-ss via wireless channels. There are interferences when MDs offload tasks to MEC-ss simultaneously over the shared wireless channel. Thereby the data transmission time is influenced for each MD. We settle queueing models for MDs and MEC-ss. Tasks from different MDs wait in a queue at each MEC-s. Therefore the waiting time is affected for each other. That is to say, MDs’ offloading strategies have impact on response time of each other. For each MD, the offloading should consider physical conditions: The computation task generation, the harvested energy arrival, the wireless channel state, the queueing time, and the power budget constraint. How to choose MEC-ss and how many should be offloaded becomes a challenging problem for each MD. First, a noncooperative game framework is established to describe the MDs’ offloading. In the game, MDs are players and the action is a vector that corresponds to the offloading amount of tasks for each MEC-s (the value is 0 for these MEC-ss that are not selected for offloading) and for local process. There are constraints on actions to comply with physical conditions. The payoff is the average response time of computation offloading for each MD that includes the communication time, the computing time, and the waiting time. Next, the best-response iterated algorithm is designed for the proposed game. Although the optimization problem of each MD is intractable, it is decomposed into a 2-step structure: one dimensional search and a sub-problem. The sub-problem is NOT convex, but it can be solved through searching over the finite solutions of KKT conditions.
In conclusion, the contributions of the paper are in three-fold.
We consider multiple heterogeneous energy harvesting MEC systems with multiple MDs and multiple MEC-ss. The randomness of computation task generation, harvested energy arrival, wireless channel state, queueing at the MEC-s, the power budget constraint, and the interference among MDs when offloading are taken into account.
A game theoretic framework is developed. We formulated a noncooperative computation offloading game for MDs, an action vector is designed to denote the offloading amount for these chosen MEC-ss. In addition, the NE is defined and its existence is analyzed.
The one-MD optimization problem is solved by a 2-step decomposition and KKT conditions. Thereafter, a NE-orienting distributive iterated best-response algorithm is derived of the game. Numerical results demonstrate the convergence and parameter effect of the proposed algorithm.
The rest of the paper is structured as follows. The system model is presented in Section 2, and we formulate the noncooperative game. In Section 3, we carry out analysis on the optimization problem of one MD and the game. Next, the NE-orienting iterated algorithm is proposed in Section 4. We show simulations and numerical results in Section 5. Finally, Section 6 concludes the whole paper.
2 System Model and Problem Formulation
As shown in Fig. 1, consider multiple heterogeneous MEC networks consisting of MDs and MEC-ss. The MD is denoted as MD , , MD . The MEC-s is denoted as MEC-s , , MEC-s . Formally, is the MD set, and represents the MEC-s set. Each MD is equipped with energy harvesting component that can extract energy from the environment (e.g., light, thermal, kinetic, magnetic field sources). Generally, the computation capability of the MEC-s surpasses that of the MD, thereby the MD prefers to offloading computation tasks to the MEC-s. An MD chooses an MEC-s for computation offloading in the light of some merits, e.g., the real-time vicinity, the wireless channel state, etc. Denote that MD offloads computation tasks to MEC-s
with probability. Apparently, . We suppose that the MDs share the same wireless frequency bandwith for computation offloading. Consider model for an MD, and assume that the computation tasks generate at MD according to a Poisson process with arrival rate . Furthermore, the stream of generated computation tasks is divided into the offloaded computation task sub-stream with rate and local processing sub-stream with rate , where . The computation tasks offloaded from MD to MEC-s constitute a Poisson stream with arrival rate . For MEC-s , the received computation tasks from MDs compose a Poisson stream with arrival rate . queueing model is erected for each MEC-s. Denote the execution requirements (e.g., number of processor cycles to be executed) of computation tasks generated at MD
as independent and identically distributed (i.i.d.) random variable. Let and be the processor’s computing speed (cycles per second, i.e., HZ) of MD and MEC-s , respectively. The amount of data to be communicated between MD and MEC-s is depicted as i.i.d. random variables . The data transmission rate between MD and MEC-s is . The time of locally processing at MD is . The processing time of offloaded computation tasks from MD to MEC-s is , where is the computing time and is the data transmission time. The processing time of computation tasks at MEC-s can be characterized by random variable with mean
and second moment
The average waiting time of computation tasks at MEC-s can be given by 
where is the utilization of MEC-s . The average response time of offloaded computation tasks from MD to MEC-s can be expressed as
The average response time of generated computation tasks on MD is given by
The channel coefficient between MD and MEC-s is , the channel bandwidth , the transmission power from MD to MEC-s is . The data transmission rate is given by , where is the noise power and is the received interference. The power consumption can be expressed by
where . The average energy consumption of data transmission from MD to MEC-s for a computation task offloading is . The average data transmission energy consumption for a computation task offloading at MD can be written as
Power consumption due to computation at MD can be given by , where is the computation energy efficiency. The average computation power consumption at MD can be described as
where is the processor utilization of MD , is the power consumption when no computation is executed. The average power consumption of communication and computation at MD can be expressed as
Assume that the harvested energy arrival of MD is i.i.d with rate . The greedy strategy of harvested energy usage, i.e., utilizing the available harvested energy preferentially, is optimal with high probability. Therefore, the average extra power that is constrained can be written as .
MD aims to minimize the average response time under the power constraint. Mathematically,
where is the power constraint on MD . MDs are self-interested and compete each other to minimize its own average response time by adjusting the computation offloading strategy. Formally, the noncooperative game of MDs can be formulated as
where is the set of players,
is the offloading strategy111In the paper, we focus on the computation allocation strategy. When the data transmission rate is added in the strategy, it corresponds to the joint allocations of computation and resource. of player , and is the payoff.
3 Problem analysis
Optimization problem of an MD is analyzed in Section 3.1. To analyze and handle the formulated problem conveniently, we first reconstruct the objective function and the constraints. The corresponding KKT conditions are derived. However, it is very difficult to solve. Next, a 2-step decomposition strategy is built, and we further reexpress the optimization problem accordingly. The optimization can be decomposed into a one-dimension search together with a reduced subproblem. In Section 3.2, game analysis is performed. The definition and existence of mixed strategy NE are investigated.
3.1 Optimization problem of MD
Let , , , , , and . can be reexpressed as
The processor utilization constraint (11c) can be expressed as
The power constraint (11d) can be written as
,, , , , and are constant with respect to , the objective and the constraints are rewritten as the functions of , explicitly and concisely. Thereafter, the KKT conditions can be obtained in Lemma 1.
Please see Appendix A. ∎
The KKT conditions are necessary for the optimal solutions. Generally, (21) is very challenging if not intractable since the optimization variables, , are coupled together. Then we have the following lemma, i.e., Lemma 2, to derive an equivalent optimization problem.
The response time minimization problem of MD , i.e., , can be equivalently reconstructed as
In , we first fix the sum of , i.e., , in the constraint. Then, we vary the sum in feasible scope. can be viewed as an intermediate variable. Thereby, can be tackled by a 2-step strategy. First, for a given , solve the following shrunken problem,222The solution of () is investigated in Section 4. i.e., .
Next, find the optimal through one-dimensional search.
3.2 Game analysis
In the paper, we consider the computation tasks and the response time in average sense, i.e., depicted by the arriving rate and average response time, respectively. Therefore mixed strategy is appropriate for the game analysis. To begin with, the mixed strategy NE of the formulated noncooperative game is described in Definition 1.
Let be the set of probability measures over the pure strategy (action) set . is a mixed strategy of MD , and denotes the set of mixed strategies of players except .
( is Cartesian Product) is a mixed strategy NE of if for and .
If for and , is a mixed strategy NE.
According to the von Neumann-Morgenstern expected payoff theory, we have
That is to say, only pure strategy deviations when determining whether a given profile is an NE. Hence, we have Property 1. ∎
Regarding the existence, we have the following Lemma.
has at least one mixed strategy NE.
The action set is nonempty and compact, the payoff function is continuous. According to , the game has a mixed strategy NE. ∎
4 Algorithm Design
In this section, we turn our attention to solve the shrunken problem at first. After that, we conceive an NE-orienting algorithm.
is a fractional programming problem, and is not convex in general. The KKT conditions can be derived in Lemma 4.
For optimization problem , the KKT conditions can be described as
where , , and are multipliers, .