## I Introduction

Developing low-latency communication policies is a crucial requirement for next generation communication networks, especially in applications pertaining to real-time status monitoring where information needs to be kept as fresh as possible at interested destinations. The age of information (AoI) metric has been recently introduced in the literature as a suitable performance metric to assess the freshness of data through basically measuring delays from receivers’ perspectives. In applications where measurement updates regarding some phenomenon are to be sent to some destination over a period of time, the AoI is defined as the time elapsed since the most recent update at the destination was generated at the source. While it is clearly optimal to keep sending updates to maintain a low AoI, this might not be feasible all the time if transmitters have limited energy budgets. Therefore, managing the available energy becomes critical. In this paper, we consider an energy harvesting two-hop network, and propose optimal update policies that minimize the AoI in offline settings, where the energy arrival information is known prior to the start of communication, and online settings, where such information is only revealed causally over time.

Minimizing the AoI has been extensively considered in the literature under various system settings and assumptions, see, e.g., [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], and also the survey in [18]. A few of these works [13, 14, 15, 16, 17] focus on multihop settings, as opposed to the single-hop settings considered in most works. Using the AoI metric to assess the performance of energy harvesting communication systems has recently gained some attention [19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34], with the main differentiating aspects in these kinds of works being: battery capacity, offline/online knowledge of the energy arrivals, and service times (times for the updates to take effect).

In this paper, we characterize age-minimal status update policies in energy harvesting two-hop networks, in which a source is communicating with a destination through the help of an intermediate relay node. We consider the setting in which both the source and the relay are equipped with infinite batteries to save their incoming energy; offline and online knowledge of the energy arrivals; and fixed service times. In the offline scenario, the goal is to minimize the average AoI by a given deadline. We first solve a single-hop version of the problem, and then use its solution to solve for the two-hop version. Specifically, we show that it is optimal for the source to transmit a new update just in time as the relay is ready to forward it to the destination. Thus, the source and the relay can be treated as one combined node, after some appropriate transformations. The solution of the offline problem follows via an inter-update balancing algorithm, which aims at uniformly spreading the updates over time, up to the extent allowed by the energy arrivals and service times. In the online scenario, the goal is to minimize the long term average AoI. Energy arrives at the source and the relay according to two independent Poisson processes. We also show in this case that it is age-minimal to treat the source and the relay as one combined node. We then propose a best effort uniform update policy with service constraints, in which time is divided into slots of equal durations that depend on the service times, and an update is transmitted at the beginning of each time slot only if both the source and the relay have enough energy, otherwise both nodes stay silent and re-attempt transmission at the beginning of the next time slot. We prove that this policy is age-minimal by showing that it achieves a lower bound on the long term average AoI almost surely.

We note that the optimality of treating the source and the relay nodes as one combined node, and its consequences on not keeping any update packets waiting in the relay’s data buffer, echoes the optimality results of last generated first served policies in the multihop setting considered in [15]. We also note that such result is in contrast to the optimality of separable policies that maximize throughput in energy harvesting two-hop networks [35, 36], where it is optimal to treat each node independently and send as many packets as possibly allowed by its own energy arrivals without considering those of the other node. In that sense, this work shows that if the metric considered is AoI, optimal policies are inseparable.

## Ii System Model and Problem Formulation

A source node acquires measurement updates from some physical phenomenon and sends them to a destination, through the help of a half-duplex relay, see Fig. 1. Both the source and the relay depend on energy harvested from nature to transmit their data, and are equipped with infinite-sized batteries to save their incoming energy. Energy arrives in packets of amounts and at the source and the relay, respectively. Update packets are of equal length, and are transmitted using fixed rates at the source and the relay. We assume that one update transmission consumes one energy packet at a given node, and hence the number of updates at a given time is equal to the minimum of the number of energy packets that arrived at the source and the relay by that time. Under a fixed rate policy, each update takes and amount of time to get through the source-relay channel and the relay-destination channel, respectively^{1}^{1}1 can be considered, for instance, equal to where is the update packet length in bits and is the transmission rate in bits/time units, where is some increasing function representing the rate-energy relationship..

The goal is to send updates as timely as possible, namely, such that the age of information (AoI) is minimized. The AoI metric at time is defined as

(1) |

where is the time stamp of the latest received update packet at the destination before time , i.e., the time at which it was acquired at the source. We define two different problem settings depending on whether the future energy arrival times are known prior to the start of the communication session, and discuss them in details next.

### Ii-a The Offline Setting

In the offline setting, we consider a communication session of duration time units, in which energy arrival times are known a priori. Without loss of generality, we assume . The objective is to minimize the following quantity:

(2) |

Source energy packets arrive at times , and relay energy packets arrive at times , where without loss of generality we assume that both the source and the relay receive energy packets, since each update consumes one energy packet in transmission from either node, and hence any extra energy arrivals at either the source or the relay cannot be used. Let and denote the transmission time of the th update at the source and the relay, respectively. We first impose the following constraints:

(3) |

representing the energy causality constraints [37] at the source and the relay, which mean that no energy packet can be used before being harvested. Next, we must have

(4) |

to ensure that the relay does not forward an update before receiving it from the source, which represents the data causality constraints [37]. We also have the service time constraints

(5) |

which ensure that there can only be one transmission at a time at the source and the relay. Hence, and represent the service (busy) time of the source and relay servers, respectively.

Transmission times at the source and the relay should also be related according to the half-duplex nature of the relay operation. For that, we must have the half-duplex constraints

(6) |

where denotes the empty set, since the relay cannot receive and transmit simultaneously. These constraints enforce that either the source transmits a new update after the relay finishes forwarding the prior one, i.e., for some ; or that the source delivers a new update before the relay starts transmitting the prior one, i.e., for some and . The latter case means that there are update packets waiting in the relay’s data buffer just before time . We prove that this case is not age-optimal. To see this, consider the example of having update packets in the relay’s data buffer waiting for service. The relay in this case has two choices at its upcoming transmission time: 1) forward the first update followed by the second one sometime later, or 2) forward the second update only and ignore the first one. These two choices yield different age evolution curves. We observe, geometrically, that under choice 2 is strictly less than that under choice 1. Since the source under choice 2 consumes an extra energy packet to send the first update unnecessarily, it should instead save this energy packet to send a new update after the first one is forwarded by the relay. Therefore, it is optimal to replace the half-duplex constraints in (6) by the following reduced ones:

(7) |

Next, observe that (5) can be removed from the constraints since it is implied by (4) and (7). In conclusion, the constraints are now those in (3), (4), and (7). Finally, we add the following constraint to ensure reception of all updates by time :

(8) |

In Fig. 2, we present an example of the age of information in a system with 3 updates. The area under the curve representing is given by the sum of the areas of the trapezoids , , and , in addition to the area of the triangle . The area of for instance is given by . The objective is to choose feasible transmission times for the source and the relay such that is minimized. Computing the area under the age curve for general arrivals, we formulate the offline problem as follows:

s.t. | ||||

(9) |

with and .

We note that the energy arrival times and , the transmission delays and , the session time , and the number of energy arrivals , are such that problem (II-A) has a feasible solution. This is true only if

(10) | ||||

(11) |

where (10) (resp. (11)) ensures that the th energy arrival time at the relay (resp. source) is small enough to allow the reception of the upcoming updates within time .

### Ii-B The Online Setting

In the online setting, future energy arrival times are not known a priori; they get revealed causally over time. We assume, however, that there is some central controller that observes both energy arrival processes at the source and the relay, and takes decisions based on these simultaneous observations^{2}^{2}2A completely decentralized system model in which the source takes scheduling decisions independently from the relay is a possible direction for future work. In this work, we focus on the centralized problem.. The energy arrival processes at the source and the relay are modeled as two independent Poisson processes of unit rate. The objective is to minimize the long term average area under the AoI curve. That is, to minimize

(12) |

Let us denote by and the energy available in the source’s and relay’s batteries, respectively, right before time , i.e., at time . Therefore, the batteries evolve as follows:

(13) | ||||

(14) |

where and denote the total number of energy arrivals in the interval

at the source and the relay, respectively, which are two independent Poisson random variables with parameter

. We now write the energy causality constraints slightly differently from the offline setting as follows:(15) |

Let the feasible set be the set of transmission times and that adhere to the constraints in (4), (7) and (13)-(15), in addition to the following at time : and . The online problem is now to characterize the following quantity:

(16) |

## Iii The Offline Problem

### Iii-a Solution Building Block: The Single-User Channel

In this subsection, we solve the single-user version of problem (II-A); namely, when the source is communicating directly with the destination. We use the solution to the single-user problem in this subsection as a building block to solve problem (II-A) in the next subsection. In Fig. 3, we show an example of the age evolution in a single-user setting. The area of is now given by . We compute the area under the age curve for general arrivals and formulate the single-user problem as follows:

s.t. | ||||

(17) |

where the second constraints are the service time constraints.

We note that reference [20] considered problem (III-A) when the transmission delay . We extend their results for a positive delay (and hence a finite transmission rate) in this subsection. We first introduce the following change of variables: ; ; and . These variables must satisfy , which reflects the dependent relationship between the new variables . This can also be seen from Fig. 3. Substituting by in problem (III-A), we get the following equivalent problem:

s.t. | ||||

(18) |

Observe that problem (III-A) is a convex problem that can be solved by standard techniques [38]. For instance, we introduce the following Lagrangian:

(19) |

where are Lagrange multipliers, with and . Differentiating with respect to and equating to 0 we get the following KKT conditions:

(20) | ||||

(21) | ||||

(22) |

along with complementary slackness conditions

(23) | ||||

(24) | ||||

(25) |

We now have the following lemmas characterizing , the optimal solution of problem (III-A). Lemmas 1 and 3 show that the sequence is non-increasing, and derive necessary conditions for it to strictly decrease. On the other hand, Lemma 2 shows that can be smaller or larger than , and derives necessary conditions for the two cases.

###### Lemma 1

For , . Furthermore, only if .

Proof: We show this by contradiction. Assume that for some we have . By (21), this is equivalent to having , i.e., , which implies by complementary slackness in (24) that . This means that , i.e., infeasible. Therefore holds. This proves the first part of the lemma.

To show the second part, observe that since holds if and only if , then either or . If , then by (24) we must have , which renders , i.e., infeasible. Therefore, cannot be positive and we must have . By complementary slackness in (23), this implies that .

###### Lemma 2

only if ; while only if , for .

Proof: The necessary condition for to be larger than can be shown using the same arguments as in the proof of the second part of Lemma 1, and is omitted for brevity. Let us now assume that is smaller than . By (20) and (21), this occurs if and only if , which implies that by complementary slackness in (24). Finally, by Lemma 1, we know that is non-increasing; since they are all bounded below by , and , then they must all be equal to .

###### Lemma 3

. Furthermore, only if at least: 1) , or 2) occurs.

The proof of Lemma 3 is along the same lines of the proofs of the previous two lemmas and is omitted for brevity.

We will use the results of Lemmas 1, 2, and 3 to derive the optimal solution of problem (III-A). To do so, one has to consider the relationship between the parameters of the problem: , , and . For instance, one expects that if the session time is much larger than the minimum inter-update time , then the energy causality constraints will be binding while the constraints enforcing one update at a time will not be, and vice versa. We formalize this idea by considering two different cases as follows.

#### Iii-A1

We first note that is the least value that can have for problem (III-A) to admit a feasible solution. In this case, the following theorem shows that the optimal solution is achieved by sending all updates back to back with the minimal inter-update time possible to allow the reception of all of them by the end of the relatively small session time .

###### Theorem 1

Let . Then, the optimal solution of problem (III-A) is given by

(26) | ||||

(27) | ||||

(28) |

Proof: We first argue that if , then . The last constraint in problem (III-A) then implies that , which is infeasible in this case. Therefore, we must have . By Lemma 2, this occurs only if for . Hence, we set , and observe that problem (III-A) in this case reduces to a problem in only one variable as follows:

s.t. | (29) |

whose solution is given by projecting the critical point of the objective function onto the feasible interval since the problem is convex [38]. This directly gives (26).

#### Iii-A2

In this case, we propose an algorithmic solution that is based on the necessary optimality conditions in Lemmas 1, 2, and 3. We first solve problem (III-A) without considering the service time constraints, i.e., assuming that the set of constraints is not active. We then check if any of these abandoned constraints is not satisfied, and optimally alter the solution to make it feasible.

Let us denote by problem (III-A) without the set of constraints , i.e., considering only the energy causality constraints. We then introduce the following algorithm to solve problem :

###### Definition 1 (Inter-Update Balancing Algorithm)

Start by computing

(30) |

where the set is indexed as , and then set

(31) |

If stop, else compute

(32) |

where the set is indexed as , and then set

(33) |

If stop, else continue with computing as above. The algorithm is guaranteed to stop since it will at most compute which is equal to by construction.

Note that while computing , if the is not unique, we pick the largest maximizer. Observe that the algorithm equalizes the ’s as much as allowed by the energy causality constraints. Let be the output of the Inter-Update Balancing algorithm and let denote the optimal solution of problem . We now have the following results:

###### Lemma 4

is a non-increasing sequence, and only if .

###### Lemma 5

, .

We note that Lemma 5 is similar to [20, Theorem 1]. In fact, the Inter-Update Balancing algorithm reduces to the optimal offline algorithm proposed in [20] when . When , some change of parameters can still show the equivalence. The next corollary now follows.

###### Corollary 1

Consider problem with the additional constraint that holds for some . Then, the optimal solution of the problem, under this condition, for time indices not larger than is given by .

Proof: This is direct by setting and , and applying the Inter-Update Balancing algorithm on the problem with a reduced number of variables .

The following theorem shows that the optimal solution of problem (III-A), , is found by equalizing the inter-update times as much as allowed by the energy causality constraints. If such equalization does not satisfy the minimal inter-update time constraints, we force it to be exactly equal to such minimum and adjust the last variable accordingly. The proof of the theorem is in Appendix -C.

### Iii-B Two-Hop Network: Solution of Problem (Ii-A)

We now discuss how to use the results of the single-user problem to solve problem (II-A). We have the following theorem:

###### Theorem 3

Proof: Let denote the objective function of problem (II-A). Differentiating with respect to , , we get , which is negative since . We also have , which is non-positive since . Thus, is decreasing in and non-increasing in . Therefore, the optimal satisfies the data causality constraints in (4) with equality for all updates so as to be the largest possible and achieve the smallest . Setting in problem (II-A) we get

(37) |

with the constraints now being

(38) | |||

(39) | |||

(40) |

We now see that minimizing subject to the above constraints is exactly the same as solving problem (III-A) after applying the change of parameters mentioned in the theorem.

Theorem 3 shows that the source should send its updates just in time as the relay is ready to forward, and no update should wait for service in the relay’s data buffer. Thus, the source and the relay act as one combined node that can send updates whenever it receives combined energy packets at times . This fundamental observation can be generalized to multi-hop networks as well. Given relays, each node should send updates just in time as the following node is ready to forward, until reaching destination.

## Iv The Online Problem

In this section, we focus on characterizing the optimal solution of problem (16). We follow a two-step approach: we first derive a lower bound on the objective function of the problem, and then propose an online-feasible policy that achieves that lower bound, which shows its optimality. Let the random variable denote the number of updates received by the destination within time . We first begin by stating an upper bound on the update rate in the limit as grows infinitely large in the following lemma:

###### Lemma 6

The following upper bound holds:

(41) |

Proof: By ignoring the energy causality constraints, and since an update can be received at the destination if and only if both the source and the relay have energy, we have

(42) |

Since both energy arrival rates at the source and the relay are unit-valued, dividing both sides of the above equation by and taking limits shows that the update rate is upper-bounded by . On the other hand, the service rate constraints imply that

(43) |

which gives another upper bound of on the update rate. Combining the two upper bounds gives the result in (41).

The above lemma’s result is quite intuitive. It shows that there can be two bottlenecks to the system’s number of updates, one due to energy constraints, and the other due to service time constraints. Determining which one is in effect depends on the value of : if it is larger than , which is the average energy arrival rate at both the source and the relay, then the system is more constrained by the service times; and if it is less than , then the system is more constrained by the energy arrival rate. This simple observation leads to a variation in the optimal update policy as we show in the sequel. In the next lemma, we use the above result to derive a lower bound on the optimal solution, , of problem (16). The proof of the lemma is in Appendix -D.

###### Lemma 7

The following lower bound holds:

(44) |

Observe that Lemma 7 reemphasizes the fact that the system’s bottleneck depends on the relationship between the total service time and the average energy arrival rate. Clearly, if , the lower bound would be given by the first term in the maximum in (44), and would be given by the second term otherwise. We will use this fact while devising the optimal policy below. Let us now define the following online update policy:

###### Definition 2 (Best Effort Uniform Update Policy with Service Constraints)

The source schedules transmission of a new status update at times: , . At , if the source and the relay have at least and units of energy, respectively, then the update is transmitted from the source and gets forwarded by the relay directly once it is received. Otherwise, if either the source or the relay has an empty battery at , then both nodes stay silent until the next scheduled transmission time .

Observe that the best effort uniform update policy with service constraints has a scheduled update rate of , which is the maximal possible update rate according to (41). Our goal now is to show that such rate is indeed achievable. This is mainly shown by proving that the failure update rate, which is given by the ratio of the number of scheduled update times in which no transmission occurs (due to either the source or the relay having an empty battery) to the total elapsed time, is negligible in the long run. This is formally proven in the next theorem, the main result of this section, whose proof is in Appendix -E.

###### Theorem 4

The long term average AoI under the best effort uniform update policy with service constraints achieves the lower bound in Lemma 7, and is therefore optimal.

We note that when , the optimal policy is equivalent to a greedy policy, in which an update is sent whenever both the source and the relay have enough energy. This is clear from the fact that the best effort uniform update policy with service constraints schedules updates every time units, i.e., back-to-back. The performance of the greedy policy, however, deteriorates when . We touch upon this note again in the numerical results section below.

## V Numerical Results

We now present some numerical examples to further illustrate our results. We start by the offline ones. A two-hop network has energy arriving at times at the source, and at the relay. A source transmission takes time unit to reach the relay; a relay transmission takes time units to reach the destination. Session time is . We apply the change of parameters in Theorem 3 to get new energy arrival times , new transmission delay , and new session time . Then, we solve problem (III-A) to get the optimal inter-update times, using the new parameters. Note that , whence the optimal solution is given by Theorem 2. We apply the Inter-Update Balancing algorithm to get . Hence, the first infeasible inter-update time occurs at (). Thus, we set: and ; ; and . We see that satisfies the conditions stated in Lemmas 1, 2, and 3.

We consider another example where energy arrives at times and , with . Applying the change of parameters in Theorem 3 we get , and hence we use the results of Theorem 1 to get . We then increase to . This is effectively according to Theorem 3, and therefore we apply Theorem 2 results. The Inter-Update Balancing algorithm gives , and hence . Since , then the optimal solution is given by (26)-(28) as .

We now present some online results. We create two independent sample paths according to unit-rate Poisson processes at the source and at the relay over the duration of time units. We then use them to compute the long term average AoI achieved by the proposed best effort uniform updating policy with service constraints as a function of the aggregate service time . We also compare this to a greedy policy, in which an update is transmitted whenever both the source and the relay have enough energy, regardless of the value of . We compute an average performance of both policies over iterations, and plot the results in Fig. 4. In the figure, we also plot the theoretical lower bound of Lemma 7. We see that the proposed policy and the lower bound are almost identical, as expected. We also see the superiority of the proposed policy in the low service time regime, i.e., when , compared to the greedy policy. In addition, as discussed after Theorem 4, the greedy policy becomes identical to the proposed policy in the high service time regime, when .

## Vi Conclusions and Future Directions

Age-minimal transmission policies have been proposed for energy harvesting two-hop networks with fixed service times. It has been shown that the optimal update policy is such that the relay’s data buffer should not contain any update packets waiting for service. Instead, updates should be transmitted from the source just in time as the relay is ready to forward them to the destination. In the offline setting, the optimal source transmission times, minimizing the average age of information by a given deadline, were found by an inter-update balancing algorithm that takes service times into consideration, as well as the knowledge of the incoming energy arrival information. In the online setting, a best effort uniform update policy with service constraints is shown to minimize the long term average age of information, in which time is uniformly divided into time slots of durations depending on service times, and an update is transmitted from the source at the beginning of each time slot only if both the source and the relay have enough energy.

The considered system model in the online setting of this paper is centralized, in which energy arrivals at both the source and the relay nodes are revealed simultaneously causally over time to some central controller that takes transmission decisions. It would be of interest to extend this work to a decentralized setting, in which both nodes take their own decisions independently of each other. Clearly, such decentralized problem is more challenging, but it may represent some practical applications where no central controller is present. Another line of extension would be to consider the finite battery case, in both centralized and decentralized scenarios, and extend the notions of threshold policies, which are optimal in single-hop settings [32, 33], to multihop settings.

### -a Proof of Lemma 4

We show this by induction. Clearly, we have by construction. Now assume that is non-increasing, and consider . We know that by construction. We now proceed by contradiction; assume that . This means that the following holds:

(45) |

or equivalently

(46) |

Next, observe that the following holds by construction when choosing :

(47) |

which is equivalent to

(48) |

This contradicts (46), and proves the first part of the lemma.

Now let us show the second part. Assume that . Then necessarily we must have for some , or else they should be equal. Therefore, by construction, we have

(49) |

This concludes the proof.

### -B Proof of Lemma 5

Let be the output of the Inter-Update Balancing algorithm and let denote the optimal solution of problem . We first show that is feasible. Let be such that . Then

Comments

There are no comments yet.