Aging Bandits: Regret Analysis and Order-Optimal Learning Algorithm for Wireless Networks with Stochastic Arrivals

by   Eray Unsal Atay, et al.

We consider a single-hop wireless network with sources transmitting time-sensitive information to the destination over multiple unreliable channels. Packets from each source are generated according to a stochastic process with known statistics and the state of each wireless channel (ON/OFF) varies according to a stochastic process with unknown statistics. The reliability of the wireless channels is to be learned through observation. At every time slot, the learning algorithm selects a single pair (source, channel) and the selected source attempts to transmit its packet via the selected channel. The probability of a successful transmission to the destination depends on the reliability of the selected channel. The goal of the learning algorithm is to minimize the Age-of-Information (AoI) in the network over T time slots. To analyze the performance of the learning algorithm, we introduce the notion of AoI regret, which is the difference between the expected cumulative AoI of the learning algorithm under consideration and the expected cumulative AoI of a genie algorithm that knows the reliability of the channels a priori. The AoI regret captures the penalty incurred by having to learn the statistics of the channels over the T time slots. The results are two-fold: first, we consider learning algorithms that employ well-known solutions to the stochastic multi-armed bandit problem (such as ϵ-Greedy, Upper Confidence Bound, and Thompson Sampling) and show that their AoI regret scales as Θ(log T); second, we develop a novel learning algorithm and show that it has O(1) regret. To the best of our knowledge, this is the first learning algorithm with bounded AoI regret.


AoI-Constrained Bandit: Information Gathering over Unreliable Channels with Age Guarantees

Age-of-Information (AoI) is an application layer metric that has been wi...

Decentralized Age-of-Information Bandits

Age-of-Information (AoI) is a performance metric for scheduling systems ...

Age of Information Optimization in Multi-Channel Network with Sided Information

We consider a discrete-time multi-channel network where the destination ...

Correlated Age-of-Information Bandits

We consider a system composed of a sensor node tracking a time varying q...

Efficient Learning-based Scheduling for Information Freshness in Wireless Networks

Motivated by the recent trend of integrating artificial intelligence int...

Learning Algorithms for Minimizing Queue Length Regret

We consider a system consisting of a single transmitter/receiver pair an...

Waiting but not Aging: Age-of-Information and Utility Optimization Under the Pull Model

The Age-of-Information (AoI) has recently been proposed as an important ...

Please sign up or login with your details

Forgot password? Click here to reset