Coordinated Reinforcement Learning for Optimizing Mobile Networks

09/30/2021
by   Maxime Bouton, et al.
0

Mobile networks are composed of many base stations and for each of them many parameters must be optimized to provide good services. Automatically and dynamically optimizing all these entities is challenging as they are sensitive to variations in the environment and can affect each other through interferences. Reinforcement learning (RL) algorithms are good candidates to automatically learn base station configuration strategies from incoming data but they are often hard to scale to many agents. In this work, we demonstrate how to use coordination graphs and reinforcement learning in a complex application involving hundreds of cooperating agents. We show how mobile networks can be modeled using coordination graphs and how network optimization problems can be solved efficiently using multi- agent reinforcement learning. The graph structure occurs naturally from expert knowledge about the network and allows to explicitly learn coordinating behaviors between the antennas through edge value functions represented by neural networks. We show empirically that coordinated reinforcement learning outperforms other methods. The use of local RL updates and parameter sharing can handle a large number of agents without sacrificing coordination which makes it well suited to optimize the ever denser networks brought by 5G and beyond.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/27/2019

Deep Coordination Graphs

This paper introduces the deep coordination graph (DCG) for collaborativ...
06/19/2020

Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) requires coordination to effic...
05/25/2021

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Recently, deep reinforcement learning (RL) algorithms have made great pr...
03/06/2022

Depthwise Convolution for Multi-Agent Communication with Enhanced Mean-Field Approximation

Multi-agent settings remain a fundamental challenge in the reinforcement...
12/19/2019

Extendable NFV-Integrated Control Method Using Reinforcement Learning

Network functions virtualization (NFV) enables telecommunications servic...
10/31/2020

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Joint Perception-Action Tasks

The purpose of this tutorial is to help individuals use the FireCommande...
03/17/2022

Strategic Maneuver and Disruption with Reinforcement Learning Approaches for Multi-Agent Coordination

Reinforcement learning (RL) approaches can illuminate emergent behaviors...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.