In a networked control and monitoring IoT-based system, it is of utmost importance to deliver timely status updates and thus keep the information fresh, for stable operation. Performance metrics using the Age of Information (AoI) and Peak AoI (PAoI) processes have first been proposed in [1, 2, 3] in continuous-time, for quantitative assessment of information freshness in status update systems. Since then, there has been a surge of interest in AoI research in terms of development of queueing models [4, 5, 6, 7, 8, 9] or AoI optimization [10, 11, 12, 13, 14]
. In the discrete-time setting, for each information source, there is an underlying stochastic process which is randomly sampled, and the sampled values are transmitted in the form of information packets towards a remote monitor via a packet-based communications network. Subsequently, for each information source, there is an individual stochastic process, called the AoI process (or sequence), which is maintained at the monitor that keeps track of the time elapsed since the generation of the last successfully received update packet. Therefore, this per-source AoI process is a cyclic process that increases in time with unit steps until the end of a cycle at which a packet reception occurs and subsequently the AoI process experiences an abrupt downward jump. On the other hand, the PAoI process is obtained by sampling the AoI sequence at the embedded epochs just before the downward jumps during each cycle of the AoI process.
The focus of this paper is on an IoT-based information update system operating in discrete-time in Fig. 1 comprising independent information sources each equipped with a sensor, a queue-server combination local to the sources, and a remote monitor. In this framework, source-, generates information packets according to a Bernoulli process with parameter
containing sensed data and a time stamp. The server is in charge of sending the information packets to the monitor via a communications network which introduces random delays, i.e., service time of packets, and the monitor immediately sends back positive acknowledgments to the server. We assume in this paper that the service times of all users are geometrically distributed with the same parameter. Particularly, we study the following three queueing disciplines employed at the server:
Non-Preemptive Bufferless (NPB) system for which one of the generated information packets at each time instant is selected uniformly at random, or randomly in short, to be placed in service when the server is idle. Otherwise, all packets are discarded.
Preemptive Bufferless (PB) system in which one of the generated information packets at each time instant (selected randomly) is always placed in service while possibly preempting the one in service.
Non-Preemptive Single Buffer with Replacement (NPSBR) system for which we have a buffer holding one information packet only and one of the generated information packets at each time instant is randomly selected to replace the one in the waiting room. If there are no packets in the buffer, this information packet is placed in service.
The main contribution of this paper is that, based on the theory of QBDs, we propose a novel analytical modeling technique to derive the exact distributions of per-source AoI and PAoI in discrete-time, for NPB, PB, and NPSBR. Although distributional results exist in the literature for continuous-time, results for discrete-time are less mature except for ones that provide average AoI and PAoI values in less general settings.
The organization of the paper is as follows. In Section 2, related work is presented. Section 3 presents QBDs. Section 4 addresses the proposed queueing model for the bufferless queueing disciplines NPB and PB as well as the single-buffer system NPSBR. Numerical examples are provided to validate the accuracy and effectiveness of the proposed queueing model in Section 5. Finally, we conclude in Section 6.
2 Related Work on Queueing Models for AoI and PAoI
The AoI concept was first introduced in  as a single-source, single-server queueing model. The case of multiple sources is then investigated in  in the same setting. Variations of this single-server queueing model in Fig. 1 for AoI has caught the attention of researchers and practitioners for modeling IoT-based status update systems in the literature [2, 17]. The majority of the existing queueing models on AoI or PAoI are for continuous-time operation.
Let us first briefly overview the literature on single-source continuous-time models. In , the mean AoI is obtained for the single-source , , and queues with infinite buffer capacity and FCFS scheduling. Although current packet-switched communication networks such as the Internet employ large buffers and FCFS scheduling at the routers, their use is shown to have adverse effects on AoI performance figures in moderate to high load regimes . The distributions of AoI and PAoI are therefore studied in  for the case of small buffers including the conventional and queues, as well as the non-preemptive LCFS queue. Exact expressions for the stationary distributions of AoI and PAoI for a very wide class of single-source information update systems are given in . The reference  obtains the exact distributions of AoI and PAoI in a bufferless system with probabilistic preemption as well as a single-buffer system that allows probabilistic replacement of the waiting packet by a newer packet arrival.
Next, we give an overview of the existing literature on multi-source status update systems in continuous-time. Mean PAoI expressions for and systems with heterogeneous service time requirements are presented in . The reference  investigates the multi-source model with FCFS as well as two disciplines: preemptive bufferless and nonpreemptive single buffer with replacement using the theory of stochastic hybrid systems (SHS) and obtain exact expressions for the mean AoI. A preemptive queue is considered in  with a common service time for all sources in which expressions for the mean AoI and PAoI are derived. The authors of  allow self-preemption in which case mean AoI expressions are derived for each source using SHS-based techniques whereas the reference  considers a two-source queueing system in which a packet waiting in the queue can be replaced only by a newly-arriving packet from the same source, again using SHS. A non-preemptive with common service times across sources is again studied by SHS in  and mean AoI expressions are derived. A more general hyper-exponential service time distribution for each class is considered in  for an non-preemptive bufferless queue to obtain an expression for the mean AoI per class.
The existing research on discrete-time AoI queueing models is less mature in comparison with continuous-time models. A discrete-time queueing model with Bernoulli arrivals and geometric service times, using FCFS and non-preemptive LCFS scheduling is presented in  with expressions for the mean AoI and PAoI values. The authors of  study the FCFS-type queue and derive explicit expressions for average AoI and PAoI and also mean AoI expressions for the discrete-time LCFS queue. For the single-source case,  considers the FCFS queue and obtains closed-form expressions of the generating functions and the stationary distributions of the AoI and the PAoI and provide a methodology for analyzing more general non-linear age functions.
3 Discrete-Time Markov Chains (DTMC) of Quasi-Birth-Death (QBD) Type
We first present notation. Uppercase bold letters are used to denote real-valued matrices. Lowercase bold (plain) letters or symbols are used to denote real-valued vectors (scalars). The notations , , and denote a matrix of zeros of size
, an identity matrix of size, and a column matrix of ones of size , respectively. When used without a subscript, we leave it to the reader to infer the size information, from the context. The function stands for the discrete-time unit step function, i.e., and is zero otherwise. The function stands for the discrete-time unit impulse function, i.e., for and is zero, otherwise.
, if its probability mass function (pmf), denoted by, is of the form :
The probability generating function (pgf) of , denoted by , is then of the form
The factorial moments of MG distributions can be found by differentiating (2) successively with respect to and substituting . Consequently, the factorial moment of an MG-distributed random variable can be found through the following expression (see ):
An infinite QBD-type DTMC
is a two-dimensional chain with , where represents the level sequence of the QBD, stands for the phase sequence, and has an irreducible probability transition matrix of the following canonical block tridiagonal form:
with , , , , , being non-negative matrices. Finite QBD chains where the level of the chain stays within a bound is outside the scope of this paper. The stationary probability vector where , of size , is the solution vector for level , is the unique solution to the following equations
The stationary solution (when it exists) has a matrix-geometric form , i.e.,
where the matrix-geometric rate matrix
(all its eigenvalues being inside the unit circle) is the unique minimal non-negative solution of the following quadratic matrix equation:
Several algorithms with computational complexity are known in the literature for obtaining the rate matrix including the logarithmic reduction procedure , or the invariant subspace algorithm using the ordered Schur decomposition , both algorithms being computationally efficient and stable. In the current paper’s numerical examples, we use the latter algorithm.
Let be the steady-state random variable associated with the level sequence of the QBD with the following expression for its pmf :
Therefore, . Another relevant random variable is the steady-state level sequence but restricted to a particular subset of the entire set of phases. For this purpose, we define to be the steady-state random variable associated with the level sequence of the QBD restricted to the particular subset . In this case,
It is not difficult to show that
where if and is zero otherwise, and
Therefore, factorial moments (hence also ordinary moments) of can easily be computed from the expression (3) for any subset .
4 Discrete-Time Queueing Modeling of AoI and PAoI
We first describe the AoI and PAoI processes (sequences) for the three queueing disciplines NPB, PB, and NPSBR, accompanied by an illustrative example. For this purpose, a successful information packet is first defined as one which is received successfully by the monitor; others those that could not start service or that were preempted while in service (this latter situation being specific to PB) are called unsuccessful packets. Let denote the arrival instant of the successful source- information packet arriving at the server and let denote the reception instant at the monitor of the successful source- information packet. We denote by the discrete-time discrete-valued random sequence representing the AoI for source- at discrete-time instant with a given initial condition .
When , is incremented by one at each time instant until the first source- successful packet reception occurring at instant at which it is again first incremented to yield the PAoI value where denotes the PAoI process for source- which is a discrete-time discrete-valued random sequence associated with the AoI just before the reception of the successful source- information packet.
Moreover, at instant , just after the packet’s reception, is set to where is the time spent in service by the successful source- information packet i.e., .
Following this packet reception, is again incremented by one at each time instant until the reception of the second source- successful packet and the AoI random sequence is obtained by repeating this pattern forever.
Discrete-time Markov modeling is quite different than that of continuous-time since in the latter, only one event can happen at a given time. However, in discrete-time, multiple events can happen at the same time instant. Therefore, the sequence of events happening at each time instant is crucial for modeling. Depending on the particular application or implementation, different event sequences might arise. Although the methodology can be easily be extended to different event sequences, we focus on the particular sequence of events at time instantwhich is described in Table 1.
|1||At the beginning of instant , the AoI values are incremented.|
|2||The service time of the packet, say source- in service, being over or not is checked, if over then the server is placed into the idle state and the AoI value is updated as if this packet turns out to be the successful source packet.|
As stated above, in our model, we assume that only one of the newcoming arrivals can be picked and processed and it is not possible to place one of the newcoming arrivals in service and another one in the waiting room in the case of NPSBR at the same time instant. We leave such extensions for future research.
Table 2 provides the sample values of the random sequences for for a two-source example with zero initial conditions for AoI values for the three queueing disciplines NPB, PB, and NPSBR. For this particular example, we assume that source- packets arrive periodically at instants , with the first five packets indexed 1a-1e having service times 5, 2, 7, 3, and 1, respectively, and source- packets also arrive periodically at instants , with the first four packets 2a-2d having service times 4, 4, 2, and 5, respectively. We assume that the packet generated by source- (source-) is to be chosen when the two sources generate packets simultaneously at (). The following explanations are given for each of the three disciplines to follow the sample evolution of the AoI and PAOI sequences.
For NBP, at , packet 1a is placed in service and is initially incremented by one at each instant until the reception of 1a at leading to . In the meantime, packet 1b is discarded at since the server is busy opon its arrival. Subsequently, is again incremented by one but rising from the value 5 (service time of 1a) until the reception of 1e at . The packets 1b and 1c are discarded since they found the server busy and packet 1d is not picked against packet 2c. Similarly, packet 2a is not picked against 1a and packet 2b is placed in service at and therefore is initially incremented by one at each instant until the reception of 2b at giving rise to . Subsequently, is again incremented by one but rising from the value 4 (service time of 2b) until the reception of 2c at .
For BP, at , packet 1a is placed in service but is preempted by 1b which is received at . Thus, is incremented by one until leading to . Packet 2b joins service at which is preempted by 1c which is also eventually preempted by the successful packet 2c received at . Therefore, is incremented by one at each instant until giving rise to . Following this, packet 1e joins service at and is received at . Thus Subsequently, is again incremented by one but rising from the value 5 (service time of 1a) until the reception of 1e at . Thus, is incremented by one until leading to while rising from the value of 2 at .
For NPSBR, at , packet 1a is placed in service and is received at whereas 1b joins the waiting room at and starts receiving service at which gets to complete at . Packet 2b joins the waiting room at and starts to receive service at . Thus, the first successful source- reception occurs at at which packet 1c starts receiving service which gets to complete at . Packet 2c joins the waiting room at but is replaced with packet 1e which joins service at and subsequently completes at .
|6||5 ( )||6||6||6||5 (||6|
|18||1 (||5||1 (||5||12||10|
We use the notation , , and , to denote the steady-state random variables associated with the processes , , and , respectively. For ease of notation, we tag a single source out of all sources, say source-, and drop the superscript while writing the steady-state pmf for the random variables , , and , respectively:
We also denote by , , and
, the corresponding cumulative distribution functions (cdf) of the random variables, , and , respectively. The goal of this paper is to devise a numerical algorithm to write the first two pmfs given in (13) and (14) or their corresponding cdfs. The pmf given in (15) is auxiliary and is to be needed for the former two pmfs for the NPSBR scenario. If the interest is on the two pmfs related to another information source- where , the same procedure can be repeated by renumbering the sources.
Our proposed methodology relies on constructing an infinite Markov chain of QBD type that produces cycles repeating forever in such a way that one cycle begins with the arrival of a successful source- packet and evolves until the reception of the next successful class- packet. The exception to this is the PB system where a cycle is to begin with the arrival of a source- packet which is not necessarily successful. The level is always incremented until the second successful source- packet reception occurs at which we transition to an auxiliary state where the level is always decremented until the level zero is hit in order to prepare for starting the next cycle. We will show that the steady-state distribution of this properly constructed QBD enables one to find the two pmfs given in (13) and (14) for each queueing discipline of interest. Recall that source- information packet generation is governed by a Bernoulli process with parameter with . We also let . The service times of all users are geometrically distributed with the same parameter and . The load is defined as .
|1||The first source- packet is in service|
|2||The service of first source- packet is over and the server is idle|
|3||The service of first source- packet is over and the second source- packet is in service|
|4||The service of first source- packet is over and a source- packet is in service where|
|5||The service of the second source- packet is over and we prepare for the next cycle|
4.1 Non-Preemptive Bufferless (NPB) Queueing System
For NPB, we propose an infinite Markov chain of QBD type characterized as in (4) with 5 phases. Table 3 describes each of the five phases used for the NPB system. We first define the pgf (probability generating function) of the number of information packet arrivals from all sources other than source 1:
Next, we define to be the probability of no packet arrivals at a time instant,
to be the probability of a source- packet to be chosen (uniformly at random) among all packet arrivals, , to be the probability of a source- packet () to be chosen among all packet arrivals. The proposed QBD evolves in the form of repetitive cycles each of which begins with the arrival of a source- packet into an empty system and continues until the reception of the next successful source- packet. At the beginning of a cycle, the level is zero and we are at phase 1. With probability , we stay in phase 1 and we transition to phase with probability while incrementing the level. While in phase 2, we stay at phase 2 until an information packet arrival occurs. Therefore, in phase 2, we transition to phase 3 with probability , or to phase 4 with probability . The level is still incremented in all these transitions at phase 2. At phase 3, we transition to phase 5 with probability when the service time of the next successful source- packet is over and we stay at phase 3, otherwise. The level is still incremented in phase 3. While in phase 4, we either stay at phase 4, or to phase 3 when the service completes and a source- packet arrival is chosen at the same instant, or to phase 2 when the service completes but there are no new packet arrivals. Again, the level is incremented in phase 4. When in phase 5, we stay at phase 5 while decrementing the level until the level zero is hit. When the level is zero, we make a transition to phase 1 with probability one while staying at level zero. This epoch is where a new cycle gets to begin. With this description, the characterizing matrices of the QBD are as follows:
is a matrix of zeros, and are matrices of zeros except for their entry which is one, and is a matrix of zeros except for its entry which is one. From the evolution of this QBD, we observe that the values that the level sequence of the proposed QBD takes in phases 2, 3, and 4, in one QBD-cycle, coincide with the sample values of the AoI process in its own cycles, as given in Table 2. Recall that an AoI cycle starts with the age taking its initial value which is the service time of a successful source- packet which is then subsequently incremented until the reception of the next successful source- packet. This observation leads us to the following main result for the NPB queueing system.
Let having the characterizing matrices as in (17) with its stationary vector for level of size denoted by being in matrix geometric form . Then, the steady-state pmf of the AoI sequence for the NPB system for source- is the same as that of which is the steady-state level of the QBD restricted to the subset Moreover, the steady-state pmf of the PAoI sequence for the NPB system for source- is the same as that of where is the steady-state level of the QBD restricted to phase 3, i.e.,
The proof is based on sample path arguments. From the evolution of this QBD, we observe that the values that the level sequence of the proposed QBD takes in phases 2, 3, and 4, in one QBD-cycle, coincide with the sample values of the AoI process in its own cycles, as given in Table 2. Similarly, one added to the values that the level sequence of the proposed QBD takes in phase 3, in one QBD-cycle, coincide with the sample values of the PAoI sequence in one AoI cycle, as given in Table 2. The pmfs of the AoI and PAoI sequences can then explicitly be written in matrix geometric form as in (12) and their factorial moments can explicitly be written as in (3). ∎
4.2 Preemptive Bufferless (PB) Queueing System
For PB, we propose an infinite Markov chain of QBD type characterized as in (4) with 5 phases, the description of the first four phases being the same as that of NPB. On the other hand, for PB, in phase 5, either the service of the second source- packet is over and we prepare for the next cycle, or the first source- packet is unsuccessful, i.e., preempted during phase 1, and after transitioning to phase 5, we prepare for the arrival of the first source- packet. This QBD evolves in the form of repetitive cycles each of which begins with the beginning of a source- packet’s service and continues until the preemption of this packet or reception of the next successful source- packet. If the source- packet is unsuccessful during phase 1, i.e., preempted, then the cycle is called an unsuccessful cycle. Otherwise, the cycle continues until the reception of the second source- information packet and is called a successful cycle. At the beginning of a cycle, the level is zero and we are at phase 1 and the service of the source- packet is about to start. To describe the operation of the QBD, we first need the following definition: , , and . The differences from the NPB system are now listed below in relation to the behavior at phases 1, 3, and 4:
At phase 1, with probability , the first source- packet gets to be preempted and we transition to phase 5 whereas the behavior at phase 5 is identical to that of the same phase in NPB. In this case, the cycle is an unsuccessful cycle. The other transitions to phases 2, 3, and 4, are identical to that of NPB.
In phase 3, if the service of source- packet successfully completes, we transition to phase 5, which occurs with probability . On the other hand, if this packet is preempted by a source-, packet, we transition to phase 4, which occurs with probability . Otherwise, we stay at phase 3.
While in phase 4, a source- packet, , is in service. We transition to phase 3 if a source- packet is chosen (with probability ) irrespective of whether the service is over or not. Additionaly, we transition to phase 2 if the service completes and there are no new arrivals, which occurs with probability .
The characterizing matrices of the proposed QBD, , are thus written as:
is matrix of zeros, and are matrices of zeros except for their entry which is one, and is a matrix of zeros except for its entry which is one. Theorem 1 stated for NPB also applies to PB with the exception that the matrix of the QBD for PB is given as in (18) as opposed to written as in (18) for NPB.
4.3 Non-Preemptive Single Buffer (NPSBR) Queueing System with Replacement
Before introducing the QBD for NPSBR as the other two queueing systems, we first need to obtain the pmf of the waiting time of successful source- packets in NPSBR. Next, we present the corresponding main result in the following theorem.
The pmf of the queue waiting time for successful source- information packets in NPSBR is a mixture of a probability mass at zero and a geometric distribution, i.e., there exist parameters such that the following hold:
Let the random sequence represent the number of information packets in the NPSBR system (service and waiting room) at instant . It is not difficult to show that the random sequence is governed by a DTMC with probability transition matrix
Let be the stationary vector of this DTMC satisfying Also let be the probability of a source- packet to be picked among all other arrivals given that the source- packet has arrived. Clearly, . Let be the success probability of a source- packet. An arriving packet joins the server with probability and joins the waiting room with probability . A packet that joins the waiting room is successful iff there are no other arrivals in a geometrically distributed interval with parameter , which occurs with probability . Therefore,
Consequently, we have the following closed-form expression for :
The conditional probability of zero queue wait, denoted by , conditioned on a successful source- packet is . Similarly, the conditional probability of a queue wait of instants, denoted by , conditioned on a successful source- packet is . Subsequently, the following choices of the two parameters and
give rise to the expression (19). ∎
|1||The first successful source- packet is in the waiting room|
|2||The first successful source- packet is in service and the waiting room is empty|
|3||The first successful source- packet is in service and there is a source- packet in the waiting room|
|4||The first successful source- packet is in service and there is a source- packet in the waiting room with|
|5||The service of first source- packet is over and we wait for a packet arrival|
|6||The second successful source- packet is in service|
|7||A source- () packet is in service and the waiting room is empty|
|8||A source- () packet is in service and and there is a source- packet in the waiting room|
|9||A source- () packet is in service and and there is a source- packet () is in the waiting room|
|10||The service of the second successful source- packet is over and we prepare for the next cycle|
For the NPSBR system, we propose an infinite Markov chain of QBD type characterized as in (4) with state-space with 10 phases. Table 4 describes each of the ten phases used the NPSBR system. The characterizing matrix of the proposed QBD is given in Eqn. (21). Moreover, is matrix of zeros, and are matrices of zeros except for their entry which is one, and is a matrix of zeros except for its entry which is and entry which is .
This QBD evolves in the form of repetitive cycles as in the previous two queueing sytems. A cycle begins with the arrival of a successful source- packet which then first waits in the waiting room (phase 1) which is then served (phase 2). Following phase 2, the QBD evolves until the completion of the next successful source- packet’s service completion (phases 3 to 9). Finally, in phase 10, we prepare for the next cycle. From the evolution of this QBD, we observe that the values that the level process of the proposed QBD takes in phases 5 to 9 in one QBD cycle, coincide with the sample values of the AoI process in its own cycles, as given in Table 2. We now present the following main result without proof for the NPSBR queueing system.
Let having the characterizing matrices as in (21) with its stationary vector for level of size denoted by being in matrix geometric form . Then, the steady-state pmf of the AoI sequence for the NPSBR system for source- is the same as that of which is the steady-state level of the QBD restricted to the subset Moreover, the steady-state pmf of the PAoI sequence for the NPSBR system for source- is the same as that of where is the steady-state level of the QBD restricted to phase 6, i.e.,
5 Numerical Results
In the first numerical example, the analytical models we have proposed for the three queueing disciplines NPB, PB, and NPSBR, are validated by simulations. For this purpose, we fix , , and for a given value of the load parameter , we set the Bernoulli arrival vector of probabilities where . The cdfs of the AoI and PAoI random sequences for the three different queueing schemes obtained with the proposed analytical model and simulations are depicted in figures 2 and 3, respectively, for and , respectively. In all the cases, we have observed a perfect match between the results obtained with the proposed analytical model and the simulation results.
In the second numerical example, we study the impact of the choice of the per-source packet generation probabilities and using the analytical model only on the system cost for a two-source system, which is given in the following form:
which allows one to give more importance to source- over source- with a proper choice of the cost parameter . When , both sources are equally important whereas when , the mean age of the second source becomes less relevant. For a given cost parameter , we do exhaustive search to find the optimum packet generation probabilities and that minimize the system cost where the minimum attainable cost is denoted by . Table 5 tabulates the optimum values , , for three different values of the service time parameter and for each of the three queueing disciplines of interest. As a second scenario, for power budgeting, we impose a bound on the overall packet generation rate while doing exhaustive search, i.e., , where is power constraint parameter since packet generation requires a certain energy consumption. In Table 6, we provide our results for the specific case of . Studying tables 5 and 6, we have the following observations:
The optimum values of and turn out to be the same for all the three queueing disciplines but the optimum cost values are not necessarily the same for each discipline.
In the lack of a power constraint, the optimum packet generation rate for source- is always one and that of source- is one only when and it decreases when decreases. Since in all these cases, there is always a packet arrival at each time instant and the two queueing disciplines NPB and NPSBR behave exactly the same under the same set of traffic parameters. In this case, the PB always outperforms the other two disciplines when the optimum packet generation rates are employed thanks to the memoryless property of the geometric service time distribution.
When there is a power constraint imposed on the packet generation rates, i.e., , the situation is very different. In this case, NPSBR always outperforms NPB and there are situations when NPSBR outperforms PB which appear to occur for lower average service times.
In this paper, we propose a discrete-time queueing model to derive the exact distributions of the AoI and PAoI sequences in a multi-source IoT-based status update system with Bernoulli packet generations and geometrically distributed service times. Three queueing disciplines are considered, namely non-preemptive bufferless, preemptive bufferless, and non-preemptive single buffer with replacement systems. The proposed method gives rise to matrix geometric expressions for the related distributions and moreover, the factorial moments of AoI and PAoI of any order are explicitly given using matrix-vector operations. Numerical examples along with simulations are presented to validate the proposed approach. Using the proposed analytical model, we also provide a numerical example on the optimum choice of the Bernoulli parameters in a practical IoT system with two sources with diverse AoI requirements and the three queueing disciplines are compared and contrasted in this setting. We have shown that the PB system yields the best performance in majority of the cases whereas it is slightly outperformed by the NPSBR discipline in scenarios with power constraints and relatively shorter average service times.