Towards Wi-Fi AP-Assisted Content Prefetching for On-Demand TV Series: A Reinforcement Learning Approach

by   Wen Hu, et al.

The emergence of smart Wi-Fi APs (Access Point), which are equipped with huge storage space, opens a new research area on how to utilize these resources at the edge network to improve users' quality of experience (QoE) (e.g., a short startup delay and smooth playback). One important research interest in this area is content prefetching, which predicts and accurately fetches contents ahead of users' requests to shift the traffic away during peak periods. However, in practice, the different video watching patterns among users, and the varying network connection status lead to the time-varying server load, which eventually makes the content prefetching problem challenging. To understand this challenge, this paper first performs a large-scale measurement study on users' AP connection and TV series watching patterns using real-traces. Then, based on the obtained insights, we formulate the content prefetching problem as a Markov Decision Process (MDP). The objective is to strike a balance between the increased prefetching&storage cost incurred by incorrect prediction and the reduced content download delay because of successful prediction. A learning-based approach is proposed to solve this problem and another three algorithms are adopted as baselines. In particular, first, we investigate the performance lower bound by using a random algorithm, and the upper bound by using an ideal offline approach. Then, we present a heuristic algorithm as another baseline. Finally, we design a reinforcement learning algorithm that is more practical to work in the online manner. Through extensive trace-based experiments, we demonstrate the performance gain of our design. Remarkably, our learning-based algorithm achieves a better precision and hit ratio (e.g., 80 the random (resp. heuristic) algorithm.


page 1

page 2

page 3

page 4


Reinforcement Learning Random Access for Delay-Constrained Heterogeneous Wireless Networks: A Two-User Case

In this paper, we investigate the random access problem for a delay-cons...

Reinforcement Learning for Improved Random Access in Delay-Constrained Heterogeneous Wireless Networks

In this paper, we for the first time investigate the random access probl...

Reinforcement Learning for Adaptive Caching with Dynamic Storage Pricing

Small base stations (SBs) of fifth-generation (5G) cellular networks are...

Multi-objective Optimization of Notifications Using Offline Reinforcement Learning

Mobile notification systems play a major role in a variety of applicatio...

Online Distributed Job Dispatching with Outdated and Partially-Observable Information

In this paper, we investigate online distributed job dispatching in an e...

Importance-Aware Fresh Delivery of Versions over Energy Harvesting MACs

We consider a scenario where multiple users, powered by energy harvestin...

Automatic Deduction Path Learning via Reinforcement Learning with Environmental Correction

Automatic bill payment is an important part of business operations in fi...

Please sign up or login with your details

Forgot password? Click here to reset