DeepAI AI Chat
Log In Sign Up

Multi-Agent Reinforcement Learning with Action Masking for UAV-enabled Mobile Communications

by   Danish Rizvi, et al.

Unmanned Aerial Vehicles (UAVs) are increasingly used as aerial base stations to provide ad hoc communications infrastructure. Building upon prior research efforts which consider either static nodes, 2D trajectories or single UAV systems, this paper focuses on the use of multiple UAVs for providing wireless communication to mobile users in the absence of terrestrial communications infrastructure. In particular, we jointly optimize UAV 3D trajectory and NOMA power allocation to maximize system throughput. Firstly, a weighted K-means-based clustering algorithm establishes UAV-user associations at regular intervals. The efficacy of training a novel Shared Deep Q-Network (SDQN) with action masking is then explored. Unlike training each UAV separately using DQN, the SDQN reduces training time by using the experiences of multiple UAVs instead of a single agent. We also show that SDQN can be used to train a multi-agent system with differing action spaces. Simulation results confirm that: 1) training a shared DQN outperforms a conventional DQN in terms of maximum system throughput (+20 for agents with different action spaces, yielding a 9 compared to mutual learning algorithms; and 3) combining NOMA with an SDQN architecture enables the network to achieve a better sum rate compared with existing baseline schemes.


page 1

page 4

page 12


Optimising Energy Efficiency in UAV-Assisted Networks using Deep Reinforcement Learning

In this letter, we study the energy efficiency (EE) optimisation of unma...

Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-agent Deep Reinforcement Learning

In the current unmanned aircraft systems (UASs) for sensing services, un...

Clustering and Power Allocation for UAV-assisted NOMA-VLC Systems: A Swarm Intelligence Approach

Integrating unmanned aerial vehicles (UAV) to non-orthogonal multiple ac...

Integrating LEO Satellites and Multi-UAV Reinforcement Learning for Hybrid FSO/RF Non-Terrestrial Networks

A mega-constellation of low-altitude earth orbit (LEO) satellites (SATs)...

Machine Learning Coupled Trajectory and Communication Design for UAV-Facilitated Wireless Networks

Augmenting wireless networks with Unmanned Aerial Vehicles (UAVs), commo...

Design of Ad Hoc Wireless Mesh Networks Formed by Unmanned Aerial Vehicles with Advanced Mechanical Automation

Ad hoc wireless mesh networks formed by unmanned aerial vehicles (UAVs) ...