Multi-Agent Deep Reinforcement Learning based Spectrum Allocation for D2D Underlay Communications

by   Zheng Li, et al.
Beijing University of Posts and Telecommunications

Device-to-device (D2D) communication underlay cellular networks is a promising technique to improve spectrum efficiency. In this situation, D2D transmission may cause severe interference to both the cellular and other D2D links, which imposes a great technical challenge to spectrum allocation. Existing centralized schemes require global information, which causes a large signaling overhead. While existing distributed schemes requires frequent information exchange among D2D users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multi-agent deep reinforcement learning is proposed, named multi-agent actor critic (MAAC). MAAC shares global historical states, actions and policies during centralized training, requires no signal interaction during execution and utilizes cooperation among users to further optimize system performance. Moreover, in order to decrease the computing complexity of the training, we further propose the neighbor-agent actor critic (NAAC) based on the neighbor users' historical information for centralized training. The simulation results show that the proposed MAAC and NAAC can effectively reduce the outage probability of cellular links, greatly improve the sum rate of D2D links and converge quickly.


A Multi-Agent Deep Reinforcement Learning based Spectrum Allocation Framework for D2D Communications

Device-to-device (D2D) communication has been recognized as a promising ...

A Multi-Agent Deep Reinforcement Learning based Spectrum Allocation Framework for D2D Underlay Communications

Device-to-device (D2D) underlay communication improves spectrum efficien...

Power Control and Channel Allocation for D2D Underlaid Cellular Networks

Device-to-Device (D2D) communications underlaying cellular networks is a...

When Distributed outperforms Centralized Scheduling in D2D-Enabled Cellular Networks

Device-to-device (D2D) communications is a promising technique for impro...

Capacity Analysis and Sum Rate Maximization for the SCMA Cellular Network Coexisting with D2D Communications

Sparse code multiple access (SCMA) is the most concerning scheme among n...

Centralized Distributed Deep Reinforcement Learning Methods for Downlink Sum-Rate Optimization

For a multi-cell, multi-user, cellular network downlink sum-rate maximiz...

Unified Scheduling for Predictable Communication Reliability in Cellular Networks with D2D Links

Cellular networks with D2D links are increasingly being explored for mis...

Please sign up or login with your details

Forgot password? Click here to reset