Reinforcement Learning-based Joint User Scheduling and Link Configuration in Millimeter-wave Networks

07/07/2022
by   Yi Zhang, et al.
0

In this paper, we develop algorithms for joint user scheduling and three types of mmWave link configuration: relay selection, codebook optimization, and beam tracking in millimeter wave (mmWave) networks. Our goal is to design an online controller that dynamically schedules users and configures their links to minimize the system delay. To solve this complex scheduling problem, we model it as a dynamic decision-making process and develop two reinforcement learning-based solutions. The first solution is based on deep reinforcement learning (DRL), which leverages the proximal policy optimization to train a neural network-based solution. Due to the potential high sample complexity of DRL, we also propose an empirical multi-armed bandit (MAB)-based solution, which decomposes the decision-making process into a sequential of sub-actions and exploits classic maxweight scheduling and Thompson sampling to decide those sub-actions. Our evaluation of the proposed solutions confirms their effectiveness in providing acceptable system delay. It also shows that the DRL-based solution has better delay performance while the MAB-based solution has a faster training process.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/27/2018

Dealing with Limited Backhaul Capacity in Millimeter Wave Systems: A Deep Reinforcement Learning Approach

Millimeter Wave (MmWave) communication is one of the key technology of t...
research
01/04/2019

Integrating Sub-6 GHz and Millimeter Wave to Combat Blockage: Delay-Optimal Scheduling

Millimeter wave (mmWave) technologies have the potential to achieve very...
research
11/17/2022

Proactive Resilient Transmission and Scheduling Mechanisms for mmWave Networks

This paper aims to develop resilient transmission mechanisms to suitably...
research
12/02/2020

Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking

This paper discusses the feasibility of beam tracking against dynamics i...
research
07/01/2022

QoE-Centric Multi-User mmWave Scheduling: A Beam Alignment and Buffer Predictive Approach

In this paper, we consider the multi-user scheduling problem in millimet...
research
06/12/2019

Cooperative Sensing in Deep RL-Based Image-to-Decision Proactive Handover for mmWave Networks

For reliable millimeter-wave (mmWave) networks, this paper proposes coop...
research
06/24/2020

Deep Reinforcement Learning for Joint Beamwidth and Power Optimization in mmWave Systems

This paper studies the joint beamwidth and transmit power optimization p...

Please sign up or login with your details

Forgot password? Click here to reset