Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph

03/02/2020
by   Chuangchuang Sun, et al.
0

The complexity of multiagent reinforcement learning (MARL) in multiagent systems increases exponentially with respect to the agent number. This scalability issue prevents MARL from being applied in large-scale multiagent systems. However, one critical feature in MARL that is often neglected is that the interactions between agents are quite sparse. Without exploiting this sparsity structure, existing works aggregate information from all of the agents and thus have a high sample complexity. To address this issue, we propose an adaptive sparse attention mechanism by generalizing a sparsity-inducing activation function. Then a sparse communication graph in MARL is learned by graph neural networks based on this new attention mechanism. Through this sparsity structure, the agents can communicate in an effective as well as efficient way via only selectively attending to agents that matter the most and thus the scale of the MARL problem is reduced with little optimality compromised. Comparative results show that our algorithm can learn an interpretable sparse structure and outperforms previous works by a significant margin on applications involving a large-scale multiagent system.

READ FULL TEXT
research
02/20/2022

PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning

Being difficult to scale poses great problems in multi-agent coordinatio...
research
08/07/2020

Towards Sample Efficient Agents through Algorithmic Alignment

Deep reinforcement-learning agents have demonstrated great success on va...
research
03/25/2022

Dealing with Sparse Rewards Using Graph Neural Networks

Deep reinforcement learning in partially observable environments is a di...
research
04/12/2023

Learning to Communicate and Collaborate in a Competitive Multi-Agent Setup to Clean the Ocean from Macroplastics

Finding a balance between collaboration and competition is crucial for a...
research
11/19/2019

Deep Tile Coder: an Efficient Sparse Representation Learning Approach with applications in Reinforcement Learning

Representation learning is critical to the success of modern large-scale...
research
06/06/2019

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents

Inspired by recent work in attention models for image captioning and que...
research
03/12/2022

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

When dealing with a series of imminent issues, humans can naturally conc...

Please sign up or login with your details

Forgot password? Click here to reset