I Introduction and Literature Review
A multihop wireless network is constituted by nodes communicating over a wireless channel. Some of the nodes, called source nodes, have data to be sent to other nodes, called receivers. In general, the data will have to be transmitted across multiple hops, over other nodes. It is necessary to develop algorithms that can ensure transmission of these data packets across the network. Any such algorithm has to take into account the topology of the network and the variability of the channels. Further, different types of data, originating from different applications, may have different Quality-of-Service (QoS) requirements, such as delay or bandwidth constraints. To design algorithms that can meet all these requirements is of interest. It is also of interest to demonstrate the performance of these algorithms in theory and by simulations.
The characterization of network performance has been approached at different angles, using various mathematical techniques. Stability of flows in a network is a common QoS requirement. Algorithms based on backpressure, such as in in 
, are throughput optimal, which means that they stabilize the network if it is possible by any other policy. Another approach is to use the framework of Markov Decision Processes.
The analysis of fluid scaling of networks was pioneered in works such as  and 
, where it was demonstrated that stability of the fluid limit of the network implies the stability of the network. Further, one may obtain bounds on moments of asymptotic values of the queues using these techniques. A comprehensive treatment of work in this direction is provided in .
Diffusion approximation of networks 
study the behaviour of the system under a scaling corresponding to the Functional Central Limit Theorem. The weak limit of the diffusion scaled systems under heavy traffic is generally a reflected Brownian motion , which under certain assumptions on the scaling rate, has a limiting stationary distribution. This distribution may be used as a proxy for the actual distribution of the system state. The diffusion approximation of the Maxweight algorithm is studied in , using properties of certain fluid scaled paths to obtain properties of the diffusion scaled paths, as in . Of these,  deals with a discrete time switch under the MaxWeight policy.
To further justify the use of the Brownian limit as a proxy for the actual system, one may try to obtain conditions in which the scaling and time limits may be interchanged. Sufficient conditions for the same are studied in  and , in the case of Jackson Networks. An important requirement for the exchange of limits in  to hold is the Lipschitz continuity of an underlying Skorohod map, which may not always hold in general.
Our main contributions in this work are summarized below.
We propose an algorithm that solves, in every slot, a weighted optimization problem. Using time varying weights that are functions of the queue lengths and mean delay requirements, the algorithm is able to dynamically cater to mean delay requirements of different flows. The function being optimized is the same as in . However, the optimization here is in every slot, and does not use the technique of discrete review. The performance of these two algorithms are same from the point of view of throughput optimality, since both result in the same set of fluid equations, and consequently are both throughput optimal.
We obtain a reflected Brownian motion (with drift) as the weak limit of the system under diffusion scaling, using techniques similar to . This Brownian motion exhibits state space collapse.
We also show that the stationary distribution of our network converges to the stationary distribution of the limiting Brownian network. This allows us to obtain the stationary distribution of our network by that of the limiting network which is explicitly available. However, our proof does not require Lipschitz continuity of the Skorohod map, unlike .
The rest of the paper is organized as follows. In Section II, we describe the system model and formulate the control policy used in the network. In Section III, we describe the two scaling regimes in which we study the network. In Section IV we prove the existence of the Brownian limit, and in Section V we prove that the stationary distribution of the limit of the scaled process is the stationary distribution of the limiting Brownian process.
Ii System Model and Control Policy
We consider a multihop wireless network (Fig. 1). The network is a connected graph with being the set of nodes and being the set of links on . The system evolves in discrete time denoted by . The links are directed, with link from node to node having a time varying channel gain at time
. Denote the channel gain vector at timeby , evolving as independent and identically distributed (i.i.d.) process across slots with distribution over a finite set . Let denote the cumulative number of slots till time when the channel state was . Let the vector of all be denoted by .
At a node , denotes the cumulative (in time) process of exogenous arrival of packets destined to node . The packets arrive as an i.i.d sequence across slots, with mean arrival rate
and variance. Let denote the vector of all . All traffic in the network with the same destination is called flow ; the set of all flows is denoted by . Each flow has a fixed route to follow to its destination. At each node there are queues, with denoting the queue length at node corresponding to flow at time . For a queue with , we have the queue evolution given by,
where is the cumulative arrival of packets by routing (i.e., arrivals from other nodes), and is the cumulative departure of packets. Let us denote, by , the cumulative number of packets of flow transmitted over link . We write,
We assume that the links are sorted into interference sets . At any time, only one link from an interference set can be active. A link may belong to multiple interference sets. We also assume that each node transmits at unit power. Then, the rate of transmission between node and node is given by an achievable rate function of and , the schedule at time .
The vector of queues at time is denoted by . Similarly we have the vectors , , and .
We want to develop scheduling policies such that the different flows obtain their end-to-end mean delay deadline guarantees. Define , and let be the set of feasible rates at time , which depends on . Our network control policy is as follows. At each , we obtain the optimal allocation ,
assuming for at least one link flow pair . If all are zero, we define the solution to be . We optimize a weighted sum of rates, with more weight given to flows with larger backlogs, with capturing the delay requirement of the flow. The weights are functions of , and denotes a desired value for the queue length of flow . We use the function
Thus, flows requiring a lower mean delay would have a higher weight compared to flows needing a higher mean delay. Flows whose mean delay requirements are not met should get priority over the other flows.The are chosen, using Little’s Law, as , where is the target end to end mean delay and is the arrival rate of flow .
Let be the number of slots till time , in which channel state was , the schedule was and flow was scheduled over . Denote the vector of all by . Define the process,
where we have (and likewise for the other processes). This process describes the evolution of the system. The state of the system at time is , which takes values in a state space . Define the capacity region as follows.
The capacity region of the network is the set of all for which a stabilizing policy exists.
Ii-a Notational Convention
We denote the set of real numbers by , and the set of integers by .
We use to denote the set of all continuous functions from to , and the set of all right continuous functions with left limits (RCLL) from to . We use to denote weak convergence. For a vector , denotes its norm (modulus). The vector of variables of the form over all and will be denoted by .
The list of symbols used in this paper is summarized below, in Table I.
|Set of nodes|
|Set of Edges|
|Set of Channel States|
|Set of Flows|
|Queue Length of flow at node|
|Cumulative Exogenous Arrivals to|
|Cumulative Departures from|
|Cumulative Arrivals to by routing|
|Cumulative number of packets of flow served on link|
|Channel gain across link|
|Cumulative slots when channel gain was|
|Time with channel , schedule , flow scheduled on|
|The process corresponding to -th scaled system|
|Arrival rate of -th system|
|Normal vector at boundary of capacity region|
Iii Two Scaling regimes
Now we describe the behaviour of under two scaling regimes, fluid and diffusion.
Iii-a Fluid Scaling
For the process , define the scaled continuous time process,
where represents the floor function. This is called the fluid scaled process. Note that the time argument on the left side is continuous, while that on the right is discrete. Whether a time argument is discrete or continuous will be generally clear from the context. Let denote the process . We have,
with the scaling in (6) being applied to each component of . Note that , and a similar notational convention holds for all the constituent functions of . The limit of , as , offers insight into the behaviour of the system under the scheduling policy in (3). The following result may be shown for our policy.
To prove this, we first show that, almost surely, a subsequential limit exists for the family . This limit
is called the fluid limit, which obeys a deterministic ordinary differential equation (o.d.e.). The proof follows by showing that this o.d.e. is globally asymptotically stable, by constructing a suitable Lyapunov function. The stability of the o.d.e. implies the stability of the associated stochastic process. The detailed proof is similar to that in, and the algorithm here and in  will have the same fluid limit equations.
Studying the fluid limit gives us insights into the stability properties of the system. However, it only proves the existence of a stationary distribution. In order to predict the behaviour of the system, one needs the stationary distribution, or some approximation to the same. However, explicitly computing the stationary distribution for our system is not feasible. Thus we define the heavy traffic regime, and the associated diffusion scaling, below. We will also show that the stationary distribution of our system process converges to that of the limiting Brownian network. This will provide us an approximation of the stationary distribution under heavy traffic, the scenario of most practical interest.
Iii-B Diffusion Scaling
Consider a sequence of systems, . Each system differs from the other in its arrival rate, . The are chosen such that, as , , and,
where is a point on the boundary of , and denotes the outer normal vector to at the point . This is known as heavy traffic scaling. We will also assume that falls in the relative interior of one of the faces of the boundary of . For this sequence of systems, we define the diffusion scaling, given by,
Let denote the process . As before, we have,
Define the system workload in the direction as,
Define the scaled process by,
Define an invariant point to be a vector that satisfies, for some ,
where is the vector of all . Then, we have the following result, which characterizes the weak convergence of the diffusion scaled processes.
Consider a sequence as described above, under heavy traffic scaling satisfying (8),and a sequence of positive integers increasing to infinity. Assume that the fluid scaled has components and that satisfy, with probability one, as
that satisfy, with probability one, as, for any , for all , , , ,
Further, assume that,
where is a non negative real number. Then, the sequence converges weakly to a reflected Brownian motion as . Further, converges weakly to .
The proof of this Theorem is detailed in the following section.
Iv Brownian Limit
The existence of the Brownian limit is demonstrated as follows. We write the scaled workload as the sum of two terms, one of which converges to a free Brownian motion, and the second as its corresponding regulating process. Together, they act as a reflected Brownian motion. Let us define, for a channel state , as the set of all feasible rate vectors. Let us denote the maximum allocation in the direction , when the channel is in state , by ,
Let be the vector , and the vector
. Define the random variables,
The random variables are i.i.d, with mean and variance given by,
Define the cumulative process,
This is the cumulative maximum possible service in the direction . We can write,
The same equation holds for , and . Define,
Thus we have,
Let us denote , and . We have the following result about the convergence of .
Assuming that the initial condition converges weakly to an invariant point, i.e,
where . Then, it follows that,
in where is a Brownian motion with drift, given by,
where is a standard Brownian motion, , and is given by (8).
Now we outline the proof of Theorem 1.
Proof of Theorem 1.
where and are identical in distribution to and . Thus is the Brownian motion given in (23). We augment this probability space to include the other components of as well. On this probability space, we will have the functions and as before. Note that, almost surely, for any sequence of increasing to infinity, properties (12) and (13) hold .
The convergence of now weakly follows if we show that, for any subsequence of , there exists a subsequence , such that, as along , almost surely,
where, almost surely, is continuous and finite for , and if , then is not a point of increase of . Then, it can be shown  that is unique, and called the regulator corresponding to , and can be represented as,
Consequently, . This is called the reflected or regulated Brownian motion corresponding to . Then it follows that converges weakly to a reflected Brownian motion as .
Therefore it suffices to show that has a limit along which satisfies the requisite properties. This is proven in the Appendix. These properties also imply that, converges weakly to . ∎
Now that we have established the existence of a limiting Brownian motion, we proceed to demonstrate how the stationary distribution of the limit of the scaled systems is equivalent to tha stationary distribution of the Brownian motion, in the next section.
V Exchange of Limits
We have the following result.
The stationary distribution of the limiting process is the limit of the stationary distributions of the constituent processes, i.e.,
where the time argument being infinity denotes the respective stationary distributions.
To prove this result, we first define a new set of fluid limit processes, given by,
Let , denote the process , and the fluid limit process obtained, for each , by taking the limit . This limit exists just as in the previous section. For each , let denote the stationary distribution of the queues. These exist because for each , the system is stable . The draining time (time for all queues to reach level zero) for the -th fluid system will be denoted by . From , we can see that is inversely proportional to the distance from the boundary of the capacity region . It is also easy to see that, due to (8), the distance to the boundary of the capacity region, which is the plane whose normal vector is , grows as . Hence we may write,
for some finite , assuming that the initial fluid level is unity.
Now, we state a sufficient condition for the sequence to be tight. Note that by writing we indicate that the initial condition of the queue is .
Assume that, for all nodes , , flows , for any , , we have, for some ,
Further, assume that there exists such that for all , we have,
Then the sequence of distributions is tight.
This result is a consequence of Theorems 3.2, 3.3 and 3.4 of . We show that the conditions of this theorem hold in our case.
Since the process is a martingale, we can use Doob’s inequality  to obtain,
where the second inequality follows from the i.i.d nature of the arrival process . Hence, (29) holds.
The bounds for and would hold if a corresponding bound holds for the processes. Let us call the slotwise allocation process as , where,
since depends on both the queue state at time , and the channel state at time . Let be the set of possible values can take. Since is finite (and consequently, ), there are only a finite set of mappings from to . This set of mappings will be denoted by . Each will take the value of one of these functions. It is easy to see that the state space of queues can be partitioned as,
where, if , we have , and the are disjoint. Now we can write,
where is the indicator function. We can further rewrite this as,
where is the set of time slots till when the queue state was in . Since the system is stationary, we can also obtain,
Thus, we may write, with ,
where depends only on . For any , along , is an i.i.d sequence. Therefore, proceeding similar to what was done for , we now obtain,
To show (32), observe that, for a particular queue , it follows from the queueing equation that,
Subtracting on either side with the corresponding fluid queue at time , we obtain,
Hence, we have,
and hence it follows for the vector process as well, with a higher constant ,
From (28), since the draining time of the fluid system with initial condition equal to one, , the fluid system with initial condition , will be zero at any time greater than . Setting , and dividing by , we get,
Since the bound is uniform over , dividing by and taking gives the result. ∎
With this result, we are ready to prove Theorem 2.
Proof of Theorem 2.
Since the are tight, any subsequence of has a convergent subsequence. Let such a limit point be . Assume that the initial conditions are distributed as . Since the systems converge to a reflected Brownian motion (RBM), the initial condition of the RBM will have distribution . Also, we have shown that finite dimensional distributions of also converge to that of . In particular, weakly converges to for any . But the distribution of is . Thus distribution of is for each . Hence is the stationary distribution of . ∎
The Brownian motion obtained as the limit of is a unidimensional Brownian motion reflected at zero, having drift . If has the stationary distribution of , we have, if ,
Vi Numerical Simulations
We verify the validity of our approximations on a star network topology (Figure 2). There are two arrival processes, one arriving at node , with node as its destination. The other arrives at node , with node as destination. We will also assume that two links which share a common node interfere with each other.
From the diffusion approximation and (LABEL:StatDistbtnExp), we can see that the mean of the Brownian motion corresponding to the queue can be approximated by the vector . The Brownian motion is a limit of the scaled process of the form . For a large , we may approximately write,
If we run the simulations for a time , we may further also approximately write . Hence, we have the approximation,
We assume that the channels are independent and identically distributed, with the distribution being uniform over the set . We consider the arrival vector , i.e., increasing along the line of unit slope. In this case . We will be looking at the total queue length of the flow . The value of is . The vector is approximately (The value of for both queues is set at ). We take . The values of the total queue length of the flow are listed in Table II (owing to symmetry both queue lengths are same), for simulation runs of length , averaged over 20 simulations. It can be seen that the approximations follow the queue length closely. Moving within a small distance of the point will require more iterations for the effects to show.
|Arrival Rate||Mean Queue Length||Approximation|
In order to demonstrate that the algorithm can satisfy different QoS requirements, we simulate the network at three points in the interior of the capacity region. The mean queue length asked from the flows is and respectively. We also pick in the expression of for the second flow to be , since it requires a tighter constraint to be met. In Table III, the first column gives the arrival rate, the second shows the target queue length for the two flows, and the final column shows the queue length obtained. We see that the end-to-end mean queue length requirement is met for both the flows till rate . At there is substantial departure. The capacity boundary is at . Thus, our algorithm can provide QoS under heavy traffic as well.
|Mean Queue Length Asked||Queue Length Obtained|
We have presented an algorithm for scheduling in multihop wireless networks that guarantees end-to-end mean delays of the packets transmitted in the network. The algorithm is throughput optimal. Using diffusion scaling, we obtain the Brownian approximation of the algorithm. We also prove theoretically that the stationary distribution of the limiting Brownian motion is the distribution of a sequence of scaled systems, and is consequently a good approximation for the stationary distribution of the original system. Using these relations, we obtain an approximation for queue lengths, and demonstrate via simulations that these are accurate.
-  M. J. Neely, E. Modiano, and C. E. Rohrs, “Dynamic power allocation and routing for time-varying wireless networks,” IEEE Journal on Selected Areas in Communications, vol. 23, no. 1, pp. 89–103, 2005.
-  R. Singh and P. Kumar, “Throughput optimal decentralized scheduling of multi-hop networks with end-to-end deadline constraints: Unreliable links,” arXiv preprint arXiv:1606.01608, 2016.
-  A. N. Rybko and A. L. Stolyar, “Ergodicity of stochastic processes describing the operation of open queueing networks,” Problemy Peredachi Informatsii, vol. 28, no. 3, pp. 3–26, 1992.
-  J. G. Dai, “On positive harris recurrence of multiclass queueing networks: a unified approach via fluid limit models,” The Annals of Applied Probability, pp. 49–77, 1995.
-  J. G. Dai and S. P. Meyn, “Stability and convergence of moments for multiclass queueing networks via fluid limit models,” IEEE Transactions on Automatic Control, vol. 40, no. 11, pp. 1889–1904, 1995.
-  S. Meyn, Control techniques for complex networks. Cambridge University Press, 2008.
-  R. J. Williams, “Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse,” Queueing systems, vol. 30, no. 1-2, pp. 27–88, 1998.
-  P. Billingsley, Convergence of probability measures. John Wiley & Sons, 2013.
-  J. Harrison, “Brownian motion and stochastic flow systems,” 1985.
-  A. L. Stolyar et al., “Maxweight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic,” The Annals of Applied Probability, vol. 14, no. 1, pp. 1–53, 2004.
-  M. Bramson, “State space collapse with application to heavy traffic limits for multiclass queueing networks,” Queueing Systems, vol. 30, no. 1-2, pp. 89–140, 1998.
-  D. Gamarnik, A. Zeevi et al., “Validity of heavy traffic steady-state approximations in generalized jackson networks,” The Annals of Applied Probability, vol. 16, no. 1, pp. 56–90, 2006.
-  A. Budhiraja and C. Lee, “Stationary distribution convergence for generalized jackson networks in heavy traffic,” Mathematics of Operations Research, vol. 34, no. 1, pp. 45–56, 2009.
-  A. K. KS and V. Sharma, “Distributed control and quality-of-service in multihop wireless networks,” in 2018 IEEE International Conference on Communications (ICC). IEEE, 2018, pp. 1–7.
-  R. F. Bass, Stochastic processes. Cambridge University Press, 2011, vol. 33.
-  A. Gut, Stopped random walks. Springer, 2009.