Model-free Reinforcement Learning for Content Caching at the Wireless Edge via Restless Bandits

02/26/2022
by   Guojun Xiong, et al.
0

An explosive growth in the number of on-demand content requests has imposed significant pressure on current wireless network infrastructure. To enhance the perceived user experience, and support latency-sensitive applications, edge computing has emerged as a promising computing paradigm. The performance of a wireless edge depends on contents that are cached. In this paper, we consider the problem of content caching at the wireless edge with unreliable channels to minimize average content request latency. We formulate this problem as a restless bandit problem, which is provably hard to solve. We begin by investigating a discounted counterpart, and prove that it admits an optimal policy of the threshold-type. We then show that the result also holds for the average latency problem. Using these structural results, we establish the indexability of the problem, and employ Whittle index policy to minimize average latency. Since system parameters such as content request rate are often unknown, we further develop a model-free reinforcement learning algorithm dubbed Q-Whittle learning that relies on our index policy. We also derive a bound on its finite-time convergence rate. Simulation results using real traces demonstrate that our proposed algorithms yield excellent empirical performance.

READ FULL TEXT
research
10/31/2022

Caching Contents with Varying Popularity using Restless Bandits

Mobile networks are experiencing prodigious increase in data volume and ...
research
05/13/2019

Deep Multi-Agent Reinforcement Learning Based Cooperative Edge Caching in Wireless Networks

The growing demand on high-quality and low-latency multimedia services h...
research
10/20/2021

Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching

Mobile edge computing (MEC) is a prominent computing paradigm which expa...
research
01/10/2021

Learning Augmented Index Policy for Optimal Service Placement at the Network Edge

We consider the problem of service placement at the network edge, in whi...
research
12/19/2017

A Reinforcement-Learning Approach to Proactive Caching in Wireless Networks

We consider a mobile user accessing contents in a dynamic environment, w...
research
04/02/2021

Hybrid Policy Learning for Energy-Latency Tradeoff in MEC-Assisted VR Video Service

Virtual reality (VR) is promising to fundamentally transform a broad spe...
research
01/09/2018

Optimal Content Replication and Request Matching in Large Caching Systems

We consider models of content delivery networks in which the servers are...

Please sign up or login with your details

Forgot password? Click here to reset