1 Introduction
The mobile data traffic demand is growing rapidly. According to the investigation
of Cisco Systems [1], the mobile data traffic is expected
to reach 24.3 exabytes per month by 2019, while it was only 2.5 exabytes per
month at the end of 2014. On the other hand, the growth rate of the mobile
network capacity is far from satisfying that kind of the demand,
which has become a major problem for wireless mobile network operators
(MNOs). Even though 5G technology is promising for providing huge wireless
network capacity [2], the development process is long and
the cost is high. Economic methods such as timedependent pricing
[3][4]
have been proposed to change users’ usage pattern, which are not userfriendly.
Up to now, the best practice for increasing mobile network capacity is to deploy
complementary networks (such as wireless LAN and femtocells), which can be quickly
deployed and are costefficient. Using such methods, part of the MUs’ traffic demand
can be offloaded from a MNO’s cellular network to the complementary network.
The process that a mobile device automatically changes its connection
type (such as from cellular network to wireless LAN) is called vertical handover [5].
Mobile data offloading is facilitated by new standards such as Hotspot 2.0 [6]
and the 3GPP Access Network Discovery and Selection Function (ANDSF)
standard [7], with which information of network (such as price and
network load) can be broadcasted to MUs in realtime. Then MUs can make offloading
decisions intelligently based on the realtime network information.
There are many works related to the wireless LAN offloading problem.
However, previous works either considered the wireless LAN offloading problem
from the network providers’ perspective without considering the MU’s
quality of service (QoS)
[8][9], or studied wireless LAN offloading
from the MU’s perspective [10][11]
[12][13],
but without taking the energy consumption as well as cost problems into consideration.
Our previous work [14] studied the wireless LAN offloading
problem from the MU’s perspective. The MU’s target was to minimize its total cost, while taking monetary cost, preference for energy consumption,
availability of MU’s mobility pattern and application’s delay tolerance into consideration.
A Markov decision process algorithm [15][16][17]
was proposed for a known MU’s mobility pattern case and a reinforcement learning
[18] based algorithm was proposed for an unknown MU’s mobility pattern case. However, [14] only considered a MU’s single flow. Actually, a MU always execute mulitple applications simultaneously with modern mobile devices that have powerful multitask abilities. Therefore, multiflow mobile data offloading problem from a MU’s perspective is more relevant and remains to be solved.In this paper, we study the wireless LAN offloading problem from a MU’s perspective considering multiflow. Each flow has different delay tolerance. The MU’s target is to minimize its total cost, which includes the monetary cost and energy consumption cost, while taking the MNO’s usage base price, the MU’s preference for energy consumption, and flows’ delay tolerance into consideration. The cost and energyaware wireless LAN offloading problem is modeled as a finitehorizon discretetime Markov decision process (FDTMDP) under the assumption that the MU’s mobility pattern is known in advance. We propose a dynamic programming based algorithm to solve the FDTMDP problem. However, the time complexity of the dynamic programming based offloading algorithm is high. Therefore, we propose a heuristic offloading algorithm with low time complexity and performance sacrifice. We conduct the simulations to verify the performance of the proposed schemes, and the simulation results show that the dynamic programming based offloading algorithm can minimize the MU’s monetary cost and save energy of the MU’s device, while the heuristic offloading algorithm has comparable performance in terms of cost minimization and energy saving for the MU.
The proposed mobile data offloading algorithms can be implemented on the MUs’ device without modification of the network system. The MUs themselves, or thirdparty application developers can utilize our work to save monetary cost and energy for the MUs.
The rest of this paper is organized as follows. Section 2 describes the related work. Section 3 illustrates the system model. Section 4 formulates the user’s wireless LAN offloading problem as discretetime finitehorizon Markov decision process and proposes a dynamic programming based algorithm. Section 5 proposes a low time complexity heuristic offloading algorithm. Section 6 illustrates the simulation and results. Finally, we conclude this paper in Section 7.
2 Related Work
Mobile data offloading has been widely studied
in the past. Gao et al. [8]
studied the cooperation among one MNO and multiple
access point owners (APOs) by utilizing the Nash bargaining
theory, and the case of multiple MNOs and multiple APOs
is studied in [9],
where double auctions were adopted.
The aforementioned papers [8][9]
considered the mobile data offloading market from the perspective
of the network without considering the MU’s experience directly.
On the other hand, papers[10][11][12][13]
have considered offloading delaytolerant traffic from the MUs’
perspective. In [10], Balasubramanian et al.
implemented a prototype system called Wiffler to
leverage delaytolerant traffic and fast switching to 3G.
Im et al. [12]
not only took a MU’s throughputdelay tradeoffs into account, but also considered the MU’s 3G budget explicitly. A MU’s mobility pattern was predicted by a secondorder Markov chain. In
[13], Cheung studied the problem of offloading delaytolerant applications for each user. A Markov decision process was formulated to minimize total data usage payment. Similar to [13], Kim et al. in [19] also utilized a Markov decision process based approach to allocate cellular network or wireless LAN data rate to maximize a MU’s satisfaction, which only depended on the MU’s wireless LAN usage.The above literature does not consider the energy consumption problem when offloading traffic from a cellular network to a complementary network. Actually, the battery life has always been a concern for smartphones. [20][21] have studied how to design an energyefficient framework for mobile data offloading. However, the trade off between throughput, delay and budget constraints have not been considered in these works. While it was shown in [11] that wireless LAN data offloading saved 55% of battery power due to the much higher data rate wireless LAN can provide, it was verified in [21] that wireless LAN could consume more energy than cellular network when wireless LAN throughput was lower. In order to clarify the contradiction, it is necessary to consider energy consumption to establish a cost and energyaware mobile data offloading scheme.
Our previous work [14] studied the wireless LAN offloading problem from a MU’s perspective. The MU’s target was to minimize its total cost under usage based pricing, while taking monetary cost, preference for energy consumption, availability of the MU’s mobility pattern and application’s delay tolerance into consideration. A Markov decision process algorithm was proposed for a known MU’s mobility pattern case and a reinforcement learning [18] based algorithm was proposed for an unknown MU’s mobility pattern case. However, [14] only considered a MU’s single flow case.
Different from the aforementioned papers, in this paper, we study a multiflow mobile data offloading problem in which a MU has multiple applications to transmit data simultaneously with different deadlines, as well as considering the MU’s monetary cost and energy consumption.
3 System Model
Since the cellular network coverage is rather high, it is assumed that the MU is always in a cellular network, but not always can access wireless LAN access points (APs). The wireless LAN APs are usually deployed at home, stations, shopping malls and so on. Therefore, we assume that wireless LAN access is locationdependent (see Fig. 1). We mainly focus on applications with data of relative large size and delaytolerance to download, for example, applications like software updates, file downloads, or emails with attachments. The MU has files to download from a remote server. Each file formulates a flow, and the set of flows is denoted as . Each flow has a deadline . T
is the deadline vector for the MU’s
flows. Please note that we only consider downlink communication in this paper. Without loss of generality, it is assumed that . We consider a slotted time system as . To simplify the analysis, we use limited discrete locations instead of infinite continous locations. It is assumed that a MU can move in possible locations, which is denoted as set . While the cellular network is available at all the locations, the availability of wireless LAN network is dependent on location . The MU has to make a decision on what network to select and how to allocate the available data rate among flows at location at time by considering total monetary cost, energy consumption and remaining time for data transmission. A MU’s mobility can be modelled by a Markovian model as in [12][13]. Therefore, the MU’s decision making problem can be modelled as a finitehorizon Markov decision process.We define the system state at as in Eq. (1)
(1) 
where is the MU’s
location index at time , which can be obtained from GPS.
is the location set.
is the vector of
remaining file sizes of all flows at time ,
for all .
is the total remaining data size for flow .
=,
is the set vector of remaining data.
The MU’s action
at each decision epoch
is to determine whether to transmit data through wireless LAN (if wireless LAN is available), or cellular network, or just keep idle and how to allocate the network data rate to flows. Therefore, the MU’s action vector is denoted as in Eq. (2)(2) 
where denotes the
vector of cellular network allocated data rates, denotes the cellular data
rate allocated to flow , and
denotes the vector of
wireless LAN network allocated data rates, and denotes
the wireless LAN rate allocated to flow .
Here the subscript and stand for cellular network
and wireless LAN, respectively.
Please note that , , …,
all can be 0 if the MU is not in the coverage area of wireless LAN AP.
Even though it is technically possible that wireless LAN and cellular network can
be used at the same time, we
assume
that the MU can not use wireless LAN and
cellular network at the same time.
We make this assumption for two reasons:
(i) If we restrict the MU to use only one network interface at the same time slot,
then the MU’s device may be used for longer time for the same amount of left battery.
(ii) Nowadays smartphones, such as an iPhone, can only use one network interface
at the same time. We can easily implement our algorithms on a MU’s device without
changing the hardware or OS of the smartphone if we have this assumption.
At time , MU may choose to use wireless LAN (if wireless LAN
is available) or cellular network, or not to use any network. If the MU chooses wireless
LAN at , the wireless LAN network allocated data rate to flow
, , is greater than 0, and the MU does not use cellular network
in this case, then = 0. On the other hand, if the MU chooses
cellular network at , the cellular network allocated data rate to
flow , , is greater than 0, and the MU does not
use wireless LAN in this case, then = 0.
, should not be greater than the remaining file
size for flow .
The sum data rate of all the flows of cellular network and wireless LAN
are denoted as and ,
respectively. and should satisfy the following
conditions.
(3) 
(4) 
and are the maximum data rates of cellular network and
wireless LAN, respectively, at each location .
Notation  Description 

, MU’s flows set.  
T  T, MU’s deadline vector. 
, the specific decision epoch of MU.  
, the location set of MU.  
, the total size of MU’s flow. .  
, vector of remaining file size.  
, state of MU.  
, MU’s location index at time .  
cellular data rate allocated to flow at time  
wireless LAN data rate allocated to flow at time  
cellular throughput in bps at location .  
wireless LAN throughput in bps at location .  
energy consumption rate of celllar network in joule/bits at location .  
energy consumption rate of wireless LAN in joule/bits at location .  
energy preference of MU at .  
MNO’s usagebased price for cellular network service.  
MU’s penalty function for remaining data at .  
MU’s energy consumption at .  
, transmission decision at .  
, MU’s policy.  
At each epoch , three factors affect the MU’s decision.

(1) the monetary cost: it is the payment from the MU to the network service provider. We assume that the network service provider adopts usagebased pricing, which is being widely used by carriers in Japan, USA, etc. The MNO’s price is denoted as . It is assumed that wireless LAN is free of charge. We define the monetary cost as in Eq. (5)
(5) 
(2) the energy consumption: it is the energy consumed when transmitting data through wireless LAN or cellular network. We denote the MU’s awareness of energy as in Eq. (6)
(6) where is the energy consumption rate of the cellular network in joule/bits at location and is the energy consumption rate of the wireless LAN in joule/bits at location . It has been shown in [21] that both and decrease with throughput, which means that low transmission speed consumes more energy when transmitting the same amount of data. According to [22], the energy consumptions for downlink and uplink are different. Therefore, the energy consumption parameters and should be differentiated for downlink or uplink, respectively. In this paper, we do not differentiate the parameters for downlink or uplink because only the downlink case is considered. Nevertheless, our proposed algorithms are also applicable for uplink scenarios with energy consumption parameters for uplink. is the MU’s preference for energy consumption at time . is the weight on energy consumption set by the MU. Small means that the MU cares less on energy consumption. For example, if the MU can soon charge his smartphone, he may set to a small value, or if the MU is in an urgent status and could not charge within a short time, he may set a large value for . = 0 means that the MU does not consider energy consumption at all in the process of data offloading, just like in [13] [19].

(3) the penalty: if the data transmission does not finish in deadline , , the penalty for the MU is defined as Eq. (7).
(7) where is a nonnegative nondecreasing function. means that the penalty is calculated after deadline .
The probabilities associated with different state changes are called transition probabilities. We denote transition probability as in Eq. (8)
(8) 
Eq. (8) shows the probability of state if action is chosen at state . It is assumed that the remaining size is independent of location change, therefore
(9) 
where
(10) 
is equal to . The MU’s probability from to is denoted as , which is assumed as known (see Assumption 1).
Assumption 1
The MU’s mobile probability to move from the current location to the next location is known in advance.
The MU’s mobility pattern can be derived from the MU’s
historical data, which has been widely studied in the
literature, such as [12].
The MU’s policy is defined as in Eq. (11)
(11) 
where is a function mapping from state
to a decision action at .
The set of is denoted as . If policy
is adopted, the state is denoted as .
The objective of the MU is to the minimize the expected
total cost (include the monetary cost and the energy consumption)
from to and penalty at with an
a optimal (see Eq. (12))
(12) 
where is the sum of the monetary cost and the energy consumption as in Eq. (13)
(13) 
Please note that the optimal action at each does not lead
to the optimal solution for the problem in Eq. (12). At each
time , not only the cost for the current time should be considered,
but also the future expected cost.
Please refer to Fig. 2 for an example of a MDP modelling
in this section. The notations used throughout this paper are summarized
as shown in Table 1.
4 Dynamic Programming Based Algorithm
The MU’s network selection and rate allocation problem has been formulated
as a standard finitehorizon discretetime Markov decision process (MDP).
The target of the MU is to choose a set of actions to minimize his cost as shown
in Eq. (12). In this section, we propose a dynamic programming
based algorithm to solve the problem in Eq. (12).
For a MDP problem, it is important to identify the optimality equation
(or Bellman equation) [23]. Denote
as the minimal expected total cost of the MU from to at state
. The Bellman equation is defined as
in Eq. (14).
(14) 
where for , , we have
(15) 
Based on the Bellman equation Eq. (14), we propose Algorithm 1. In the optimal policy calculation phase, the optimal policy is calculated by backward induction from epoch to 1, where is the granularity of the total data size . Then, the MU’s offloading data policy is decided in each slot in the offloading data transmission phase. It is obvious that the time complexity of Algorithm 1 is .
Theorem 1
The policy generated in Algorithm 1 is the problem (12)’s optimal solution.
Proof: It is obvious according to the principle of optimality defined in [23].
Q.E.D
Algorithm 1: Dynamic Programming Based Algorithm  

1:  Optimal Policy Calculation Phase 
2:  Set , , by Eq. (7) 
3:  Set := 
4:  while : 
5:  for : 
6:  Set =0 
7:  for : 
8:  Calculate using Eq. (14) 
9:  Set := 
10:  Set := 
11:  Set := 
12:  end for 
13:  end for 
14:  Set := 
15:  end while 
16:  The optimal policy is generated for the following offloading data transmission phase 
17:  
18:  Offloading Data Transmission Phase 
19:  Set , 
20:  while and : 
21:  is determined from GPS 
22:  Set action according to (the optimal policy) 
25:  Set 
26:  end if 
27:  Set 
28:  end while 
5 Low Time Complexity Heuristic Offloading Algorithm
A dynamic programming based mobile offloading algorithm (Algorithm 1) has been proposed in Sect. 4, and Theorem 1 guarantees the optimality of this algorithm. However, time complexity of Algorithm 1 is rather high. Furthermore, the Optimal Policy Calculation Phase of Algorithm 1 should be performed in advance, which means that Algorithm 1 is an offline algorithm. Therefore, two questions may arrise.

Is there a low time complexity algorithm solution for the MU’s problem in Eq. (12)?

How to generate the MU’s policy in a realtime manner without calculations in advance as in Algorithm 1?
Algorithm 2: Low Time Complexity Heuristic Offloading Algorithm  

1:  At time slot 
2:  Input: T, , , 
3:  for T: 
4:  if : 
5:  Add to deadline remain list R 
6:  Set = 
7:  else: 
8:  Set 
9:  Add to rate allocation weight list 
10:  end if 
11:  end for 
13:  Normalize to 
13:  Normalize to 
12:  =multiply() //multiply by element 
13:  Normalize to 
14:  if wireless LAN access is available at location and speed is greater than : 
15:  Allocate wireless LAN data rate to each flow according to weight list . 
16:  is determined 
17:  else if : 
18:  Allocate cellular data rate to each flow according to weight list . 
19:  is determined 
20:  end if 
21:  Output: ) 
In this section, we try to answer the aforementioned two questions and thus avoid the problems posed by the dynamic programming based Algorithm 1 in Sect. 4. An online low time complexity heuristic offloading algorithm is proposed as shown in Algorithm 2 by the following arguments:

(i) a flow with a earlier deadline should have a higher priority;

(ii) the more remaining file size, the higher is the priority of the flow;

(iii) the wireless LAN network should have a priority since it has a lower price than the cellular network;

(iv) when the flow deadline is approaching, the cellular network should also be used to try to finish the data transmission, without waiting for access to a wireless LAN network;

(v) a low speed wireless LAN network, which consumes a lot of energy for data transmission, should be ignored to save energy if the MU concerns about the energy consumption;
We briefly explain Algorithm 2 below.
The main task is
to first calculate the allocation weight W,
then allocate the wireless LAN data rate or cellular data rate based
on the calculated allocation weight. The inputs of the algorithm
are deadline vector T,
deadline threshold (which will be explained later), location , and
remaining file size vector b. The flow with the least remaining
deadline has the highest priority, which is calculated as weight
= . Here is the deadline for flow .
is the weight list for deadlines, which reflects the
aforementioned rationale (i).
Considering argument (ii) above,
the weight list for deadlines should be
multiplied by the remaining
file size vector after normalization
( is the normalization of
and is the normalization of ).
The result of multiplication is denoted as W. The reason why
normalization is necessary is that the weight for a deadline and the remaining file
size are of different scales. While wireless LAN has the priority, we have to use
wireless LAN when possible. However, if the speed of a wireless LAN is lower than a
threshold , the wireless LAN should not be used because it is
energyconsuming (rationale (v)). Please note that is a parameter
that is determined by the MU’s energy preference .
If the MU is concerned about energy consumption (high ),
the MU will eliminate low speed APs by setting a high
threshold .
If there is no wireless LAN, the MU has to wait for
wireless LAN without using cellular network. Yet if the least remaining time for data
transmission is
less than threshold ,
the cellular network also should be selected for data transmission (rationale (iv)).
It is obvious that the time complexity of Algorithm 2 is ,
which is much lower than that of Algorithm 1 and there is no offline calculation phase
for Algorithm 2, therefore, the decision is made in an online manner.
6 Performance Evaluation
In this section, the performances of our dynamic programming
based algorithm (Proposed DP) and heuristic offloading
(Proposed Heuristic) algorithm are evaluated
by comparing them with a Baseline algorithm that use
wireless LAN AP to offload traffic whenever possible and the
algorithm called DAWN in [13].
We developed a simulator with Python 2.7, which can be downloaded
from the following URL link: https://github.com/aqian2006/OffloadingMDP.
A four by four grid is used in simulation.
Therefore, is 16. Wireless LAN APs are randomly
deployed in locations. The cellular usage price is
assumed as 1.5 yen/Mbyte. means
that the probability that the MU stays in the same place
from time to is 0.6. And the MU moves to the neighbour
location with equal probability, which can be calculated as
.
The average Wireless LAN throughput
is assumed as 15 Mbps^{1}^{1}1We tested
repeatedly with an iPhone 5s on the public wireless LAN APs
of one of the biggest Japanese wireless carriers.
The average throughput was 15 Mbps. , while
average cellular network throughput is 10 Mbps^{2}^{2}2We
also tested with an iPhone 5s on one of the biggest Japanese wireless
carriers’ cellular network. We use the value 10 Mbps for average
cellular throughput.
. We generate wireless LAN throughput for each AP from a truncated normal distribution, and the mean and standard deviation are assumed as 15Mbps and 6Mbps respectively. The wireless LAN throughput is in the range [9Mbps, 21Mbps]. Similarly, we generate cellular throughput from a truncated normal distribution, and the mean and standard deviation are assumed as 10Mbps and 5Mbps respectively. The cellular network throughput is in range [5Mbps, 15Mbps].
in Algorithm 1 is assumed as 1 Mbits. Time for each epoch is 1 seconds. The penalty function is assumed as =. Please refer to Table III for the parameters used in the simulation.Throughput (Mbps)  Energy (joule/Mb) 

11.257  0.7107 
16.529  0.484 
21.433  0.3733 
Because the energy consumption rate is a decreasing function of throughput, we have the sample data from [24] (see Table II). We then fit the sample data by a exponential function as shown in Fig. 3. We also made a new energythroughput function as , which is just lower than . We basically use if we do not explicitly point out. Please note that the energy consumption rate of cellular and wireless LAN may be different for the same throughput, but we assume they are the same and use the same fitting function as in Fig. 3.
Parameters  Value 

16  
B  Mbits 
Number of wireless LAN APs  8 
1 Mbits  
time slot  1 seconds 
average of  10 Mbps 
standard deviation of  5 Mpbs 
average of  15 Mbps 
standard deviation of  6 Mpbs 
0.6  
(10.6)/#neigbour locations  
1.5 yen per Mbyte  
=  
In Fig.4, our proposed DP and heuristic algorithms are compared to the DAWN algorithm in [13] in terms of the MU’s energy consumption. Since [13] only considered a singleflow case, we also apply our algorithms to a singleflow. It is shown that the larger the MU’s energy preference, the lower the energy consumed for our MDP and heuristic algorithms. The energy consumption of the DAWN algorithm is higher than that of our algorithm. The heuristic algorithm is not optimal, but it is close to the optimal result of proposed DP algorithm. The reasons is that energy consumption was not considered in the DAWN algorithm, while our proposed DP and heuristic algorithms have taken energy consumption into consideration and tried to minimize total energy consumption.
Fig.5 and shows the comparison of monetary cost among Baseline, Proposed Heuristic, and Proposed DP algorithms with different number of flows. The monetary cost of all three algorithms increases with the number of flows. The monetary cost of Proposed DP is lower than Baseline, while Proposed Heuristic is close to Proposed DP. The reason is that in Baseline, data are downloaded whenever there is a network (cellular or wireless LAN) available, without considering the monetary cost by using the cellular network. In Proposed DP and Proposed Heuristic, whether the cellular network is used depends on the remaining data to download and the deadline. If there are only relatively few remaining data and enough time left until the deadline, our proposed algorithms will choose to wait for a cheap wireless LAN to download data.
Fig.6 shows the comparison of the energy consumption among Baseline, Proposed Heuristic, and Proposed DP algorithms with different number of flows. Two energythroughput functions and are used. The performance of Proposed DP algorithm is the best, but the Proposed Heuristic algorithm shows small differences with that of Proposed DP algorithm with either or . For Proposed DP/Proposed Heuristic/Baseline, the energy consumption under is much higher than that under . The reason is that the energy consumption for a certain throughput is higher under than that under . Our Proposed DP algorithm consumes the least energy since we attempt to minimize the total energy consumption by formulating a MDP problem.
Fig.7 shows the comparison of monetary cost among Baseline, Proposed Heuristic, and Proposed DP algorithms with different
number of APs.
It can be seen that the monetary cost of Proposed DP algorithm is lowest, and the baseline algorithm is highest.
While the monetary cost of the Proposed Hueristic algorithm is between
that of Baseline, it is much closer to the Proposed DP algorithm.
With a large number of wireless LAN APs deployed, the chance
of using cheap wireless LAN increases. Therefore, the MU can reduce his monetary
cost by using cheap wireless LAN. Therefore, all three algorithms’ monetary
costs decreases with the number of APs.
Fig.8 shows how the MU’s energy consumption changes with
the number of deployed APs under the two energythroughput functions and .
Similar to Fig.6, the performance of Proposed DP algorithm is the best with either or .
It shows that the energy consumptions of all three algorithms just slightly decrease
with the number of APs. The reason is that the energy consumption depends
on the throughput. The larger the throughput, the lower is the energy consumption.
With large number of wireless LAN APs, the MU has more chance to use wireless LAN
with high throughput since the average throughput of a wireless LAN is assumed
as higher than that of cellular network (see Table III).
Fig.9 shows the finish rate comparison among Basedline, Proposed DP and the Proposed Heuristic algorithm with different number of wireless LAN APs.
Here, finish rate is defined as the ratio of the number of flows with finished transmission to the total number of flows started. Even though there are penalties
for flows’ remaining data, not all the flows can be finished before their deadlines.
Finish rate of Proposed DP and the Proposed Heuristic algorithms increases with the number of wireless LAN APs deployed.
The reason is that a large number of cheap and high throughput
wireless LAN APs decreases the overall download time.
There are two limitations for our proposed DP and heuristic algorithms: (i) the proposed DP algorithm has a very high timecomplexity, therefore it takes time to get the optimal policy for the MU. Therefore, we proposed a low timecomplexity heuristic algorithm for the MU. (ii) The heuristic algorithm can compute the policy very fast, and the simulation results have shown that the performance is comparable with our optimal DP algorithm. But we have not theoretically proofed yet that the heuristic algorithm is optimal or near optimal.
7 Conclusion
In this paper, we studied a multiflow mobile data offloading problem
in which a MU has multiple applications that want to download data simultaneously
with different deadlines. We formulated the wireless LAN
offloading problem as a finitehorizon discretetime Markov decision
process.
A dynamic programming based offloading algorithm was proposed
and its time complexity was analyzed.
Analysis results showed that the
time complexity of the algorithm is rather high. We proposed a
low time complexity heuristic offloading algorithm.
Extensive simulations have shown that the DP algorithm
had the lowest cost, while the heuristic algorithm had comparable
performance as that of DP algorithm.
This work assumes that the MNO adopts
usagebased pricing, in which the MU paid for the MNO in proportion with data usage.
In the future, we will evaluate other usagebased pricing
variants like tiered data plan, in which the payment of the MU is
a step function of data usage. And we will also use timedependent pricing
(TDP) we proposed in [3][4], without changing
the framework and algorithms proposed in this paper.
We also assume that the MU can only use one network interface at most,
either cellular network or wireless LAN, at each time. In
future work, we will relax this assumption to see
how the energy consumption and monetary cost will be in this case.
Another assumption we have made is that the MU’s mobile probability
from one place to another is known. It is reasonable if the MU moves
in a certain pattern, for example, people may commute from home
to work by the same train at the same time in weekdays. In our
future work, we would also like to consider the case
wherein the probability of the MU’s movement from one place to another is unknown.
The Possible solution is to utilize learning technology to predict the MU’s mobile probability.
Acknowledgements
This work is part of the GrantinAid for Young Scientists (B) research programme with grant number 16K18109, which is financed by the Japan Society for the Promotion of Science (JSPS).
References
 [1] CiscoSystems, “Cisco visual networking index: Global mobile data traffic forecast update, 20142019,” Feb. 2015.
 [2] Q. C. Li, H. Niu, A. T. Papathanassiou, and G. Wu, “5G network capacity: Key elements and technologies,” IEEE Veh. Technol. Mag., vol. 9, no. 1, pp. 71–78, March 2014.
 [3] C. Zhang, B. Gu, K. Yamori, S. Xu, and Y. Tanaka, “Duopoly competition in timedependent pricing for improving revenue of network service providers,” IEICE Trans. Commun., vol. E96B, no. 12, pp. 2964–2975, Dec. 2013.
 [4] ——, “Oligopoly competition in timedependent pricing for improving revenue of network service providers with complete and incomplete information,” IEICE Trans. Commun., vol. E98B, no. 1, pp. 30–32, Jan. 2015.
 [5] J. MárquezBarja, C. T. Calafate, J.C. Cano, and P. Manzoni, “Review: An overview of vertical handover techniques: Algorithms, protocols and tools,” Comput. Commun., vol. 34, no. 8, pp. 985–997, June 2011.
 [6] CiscoSystems, “The future of hotspots: Making WiFi as secure and easy to use as cellular,” White Paper, 2011.
 [7] AlcatelandBritishTelecommunications, “WiFi roaming – building on andsf and hotspot2.0,” White Paper, 2012.
 [8] L. Gao, G. Iosifidis, J. Huang, L. Tassiulas, and D. Li, “Bargainingbased mobile data offloading,” IEEE J. Sel. Areas Commun., vol. 32, no. 6, pp. 1114–1125, June 2014.
 [9] G. Iosifidis, L. Gao, J. Huang, and L. Tassiulas, “A doubleauction mechanism for mobile dataoffloading markets,” IEEE/ACM Trans. Netw., vol. 22, no. 4, pp. 1271–1284, Aug. 2014.
 [10] A. Balasubramanian, R. Mahajan, and A. Venkataramani, “Augmenting mobile 3g using WiFi,” in Proc. 8th international conference on Mobile systems, applications, and services (MobiSys 2010), June 2010, pp. 209–222.
 [11] K. Lee, J. Lee, Y. Yi, I. Rhee, and S. Chong, “Mobile data offloading: How much can WiFi deliver?” IEEE/ACM Trans. Netw., vol. 21, no. 2, pp. 536–550, April 2013.
 [12] Y. Im, C. JoeWong, S. Ha, S. Sen, T. . Kwon, and M. Chiang, “AMUSE: Empowering users for costaware offloading with throughputdelay tradeoffs,” in Proc. IEEE Conference on Computer Communications (INFOCOM 2013), April 2013, pp. 435–439.
 [13] M. H. Cheung and J. Huang, “DAWN: Delayaware WiFi offloading and network selection,” IEEE J. Sel. Areas Commun., vol. 33, no. 6, pp. 1214 – 1223, June 2015.
 [14] C. Zhang, B. Gu, Z. Liu, K. Yamori, and Y. Tanaka, “A reinforcement learning approach for cost and energyaware mobile data offloading,” Proc. 16th AsiaPacific Network Operations and Management Symposium (APNOMS 2016), Kanazawa, Japan, pp. 1–6, Oct. 2016.
 [15] Z. Liu, C. Zhang, M. Dong, B. Gu, Y. Ji, and Y. Tanaka, “Markovdecisionprocessassisted consumer scheduling in a networked smart grid,” IEEE Access, vol. 5, pp. 2448–2458, March 2017.
 [16] Z. Liu, G. Cheung, and Y. Ji, “Distributed markov decision process in cooperative peertopeer repair for WWAN video broadcast,” Proc. 2011 IEEE International Conference on Multimedia and Expo (ICME 2011), Barcelona, Spain, pp. 1–6, July 2011.
 [17] ——, “Optimizing distributed source coding for interactive multiview video streaming over lossy networks,” IEEE Trans. Circuits and Systems for Video Technology, vol. 23, no. 10, pp. 1781–1794, Oct. 2013.
 [18] P. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.
 [19] Y. Kim, J. Lee, J. Jeong, and S. Chong, “Multiflow rate control in delayed WiFi offloading systems,” in Proc. 2016 International Conference on Information Networking (ICOIN 2016), Washington, DC, USA, Jan. 2016.
 [20] N. Ristanovic, J.Y. L. Boudec, A. Chaintreau, and V. Erramilli, “Energy efficient offloading of 3g networks,” in Proc. 2011 IEEE 8th International Conference on Mobile Adhoc and Sensor Systems (MASS 2011), Oct. 2011, pp. 202–211.
 [21] A. Y. Ding, B. Han, Y. Xiao, P. Hui, A. Srinivasank, M. Kojo, and S. Tarkoma, “Enabling energyaware collaborative mobile data offloading for smartphones,” in Proc. 10th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks (SECON 2013), June 2013, pp. 487–495.
 [22] N. Balasubramanian, A. Balasubramanian, and A. Venkataramani, “Energy consumption in mobile phones: a measurement study and implications for network applications,” Proc. 9th ACM SIGCOMM Conference on Internet Measurement (IMC 2009), Chicago, Illinois, USA, pp. 280–293, July 2009.
 [23] R. Bellman, Dynamic Programming, Princeton University Press, 1957.
 [24] A. Murabito, “A comparison of efficiency, throughput, and energy requirements of wireless access points,” Report of InterOperability Laboratory, University of New Hampshire, March 2009. [Online]. Available: http://www.cisco.com/c/en/us/solutions/collateral/serviceprovider/visualnetworkingindexvni/white{_}paper{_}c11520862.pdf
Comments
There are no comments yet.