Multi-Agent Deep Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks with Imperfect Channels

03/25/2020
by   Yiding Yu, et al.
0

This paper investigates a futuristic spectrum sharing paradigm for heterogeneous wireless networks with imperfect channels. In the heterogeneous networks, multiple wireless networks adopt different medium access control (MAC) protocols to share a common wireless spectrum and each network is unaware of the MACs of others. This paper aims to design a distributed deep reinforcement learning (DRL) based MAC protocol for a particular network, and the objective of this network is to achieve a global α-fairness objective. In the conventional DRL framework, feedback/reward given to the agent is always correctly received, so that the agent can optimize its strategy based on the received reward. In our wireless application where the channels are noisy, the feedback/reward (i.e., the ACK packet) may be lost due to channel noise and interference. Without correct feedback, the agent (i.e., the network user) may fail to find a good solution. Moreover, in the distributed protocol, each agent makes decisions on its own. It is a challenge to guarantee that the multiple agents will make coherent decisions and work together to achieve the same objective, particularly in the face of imperfect feedback channels. To tackle the challenge, we put forth (i) a feedback recovery mechanism to recover missing feedback information, and (ii) a two-stage action selection mechanism to aid coherent decision making to reduce transmission collisions among the agents. Extensive simulation results demonstrate the effectiveness of these two mechanisms. Last but not least, we believe that the feedback recovery mechanism and the two-stage action selection mechanism can also be used in general distributed multi-agent reinforcement learning problems in which feedback information on rewards can be corrupted.

READ FULL TEXT

page 1

page 10

research
12/01/2017

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks

This paper investigates the use of deep reinforcement learning (DRL) in ...
research
07/07/2022

Learning-based Autonomous Channel Access in the Presence of Hidden Terminals

We consider the problem of autonomous channel access (AutoCA), where a g...
research
04/06/2021

Distributed Deep Reinforcement Learning for Collaborative Spectrum Sharing

Spectrum sharing among users is a fundamental problem in the management ...
research
10/11/2019

Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks

This paper investigates a new class of carrier-sense multiple access (CS...
research
10/16/2018

Carrier-Sense Multiple Access for Heterogeneous Wireless Networks Using Deep Reinforcement Learning

This paper investigates a new class of carrier-sense multiple access (CS...
research
02/28/2023

On Learning Intrinsic Rewards for Faster Multi-Agent Reinforcement Learning based MAC Protocol Design in 6G Wireless Networks

In this paper, we propose a novel framework for designing a fast converg...
research
04/29/2021

Medium Access using Distributed Reinforcement Learning for IoTs with Low-Complexity Wireless Transceivers

This paper proposes a distributed Reinforcement Learning (RL) based fram...

Please sign up or login with your details

Forgot password? Click here to reset