Reinforcement Learning for Adaptive Caching with Dynamic Storage Pricing

by   Alireza Sadeghi, et al.
University of Minnesota

Small base stations (SBs) of fifth-generation (5G) cellular networks are envisioned to have storage devices to locally serve requests for reusable and popular contents by caching them at the edge of the network, close to the end users. The ultimate goal is to shift part of the predictable load on the back-haul links, from on-peak to off-peak periods, contributing to a better overall network performance and service experience. To enable the SBs with efficient fetch-cache decision-making schemes operating in dynamic settings, this paper introduces simple but flexible generic time-varying fetching and caching costs, which are then used to formulate a constrained minimization of the aggregate cost across files and time. Since caching decisions per time slot influence the content availability in future slots, the novel formulation for optimal fetch-cache decisions falls into the class of dynamic programming. Under this generic formulation, first by considering stationary distributions for the costs and file popularities, an efficient reinforcement learning-based solver known as value iteration algorithm can be used to solve the emerging optimization problem. Later, it is shown that practical limitations on cache capacity can be handled using a particular instance of the generic dynamic pricing formulation. Under this setting, to provide a light-weight online solver for the corresponding optimization, the well-known reinforcement learning algorithm, Q-learning, is employed to find optimal fetch-cache decisions. Numerical tests corroborating the merits of the proposed approach wrap up the paper.


page 1

page 12


Adaptive Caching via Deep Reinforcement Learning

Caching is envisioned to play a critical role in next-generation content...

Distributed Network Caching via Dynamic Programming

Next-generation communication networks are envisioned to extensively uti...

RLCache: Automated Cache Management Using Reinforcement Learning

This study investigates the use of reinforcement learning to guide a gen...

Towards Wi-Fi AP-Assisted Content Prefetching for On-Demand TV Series: A Reinforcement Learning Approach

The emergence of smart Wi-Fi APs (Access Point), which are equipped with...

Phoebe: Reuse-Aware Online Caching with Reinforcement Learning for Emerging Storage Models

With data durability, high access speed, low power efficiency and byte a...

A rolling-horizon dynamic programming approach for collaborative caching

In this paper, we study the online collaborative content caching problem...

Random-Mode Frank-Wolfe Algorithm for Tensor Completion in Wireless Edge Caching

Wireless edge caching is a popular strategy to avoid backhaul congestion...

Please sign up or login with your details

Forgot password? Click here to reset