DeepAI AI Chat
Log In Sign Up

Fast reinforcement learning for decentralized MAC optimization

05/17/2018
by   Eleni Nisioti, et al.
0

In this paper, we propose a novel decentralized framework for optimizing the transmission strategy of Irregular Repetition Slotted ALOHA (IRSA) protocol in sensor networks. We consider a hierarchical communication framework that ensures adaptivity to changing network conditions and does not require centralized control. The proposed solution is inspired by the reinforcement learning literature, and, in particular, Q-learning. To deal with sensor nodes' limited lifetime and communication range, we allow them to decide how many packet replicas to transmit considering only their own buffer state. We show that this information is sufficient and can help avoiding packets' collisions and improving the throughput significantly. We solve the problem using the decentralized partially observable Markov Decision Process (Dec-POMDP) framework, where we allow each node to decide independently of the others how many packet replicas to transmit. We enhance the proposed Q-learning based method with the concept of virtual experience, and we theoretically and experimentally prove that convergence time is, thus, significantly reduced. The experiments prove that our method leads to large throughput gains, in particular when network traffic is heavy, and scales well with the size of the network. To comprehend the effect of the problem's nature on the learning dynamics and vice versa, we investigate the waterfall effect, a severe degradation in performance above a particular traffic load, typical for codes-on-graphs and prove that our algorithm learns to alleviate it.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/02/2021

Towards Multi-agent Reinforcement Learning for Wireless Network Protocol Synthesis

This paper proposes a multi-agent reinforcement learning based medium ac...
10/05/2021

A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing

The increasing number of wireless devices operating in unlicensed spectr...
05/24/2021

Decentralized, Hybrid MAC Design with Reduced State Information Exchange for Low-Delay IoT Applications

We consider a system of several collocated nodes sharing a time slotted ...
04/03/2020

Multi-agent Reinforcement Learning for Networked System Control

This paper considers multi-agent reinforcement learning (MARL) in networ...
03/29/2021

Robust Reinforcement Learning under model misspecification

Reinforcement learning has achieved remarkable performance in a wide ran...
10/30/2021

Throughput and Latency in the Distributed Q-Learning Random Access mMTC Networks

In mMTC mode, with thousands of devices trying to access network resourc...
01/25/2019

Deep Learning-aided Application Scheduler for Vehicular Safety Communication

802.11p based V2X communication uses stochastic medium access control, w...