Neurosymbolic Transformers for Multi-Agent Communication

by   Jeevana Priya Inala, et al.

We study the problem of inferring communication structures that can solve cooperative multi-agent planning problems while minimizing the amount of communication. We quantify the amount of communication as the maximum degree of the communication graph; this metric captures settings where agents have limited bandwidth. Minimizing communication is challenging due to the combinatorial nature of both the decision space and the objective; for instance, we cannot solve this problem by training neural networks using gradient descent. We propose a novel algorithm that synthesizes a control policy that combines a programmatic communication policy used to generate the communication graph with a transformer policy network used to choose actions. Our algorithm first trains the transformer policy, which implicitly generates a "soft" communication graph; then, it synthesizes a programmatic communication policy that "hardens" this graph, forming a neurosymbolic transformer. Our experiments demonstrate how our approach can synthesize policies that generate low-degree communication graphs while maintaining near-optimal performance.


Low-Bandwidth Communication Emerges Naturally in Multi-Agent Learning Systems

In this work, we study emergent communication through the lens of cooper...

Multi-agent Communication with Graph Information Bottleneck under Limited Bandwidth

Recent studies have shown that introducing communication between agents ...

Multi-Agent Off-Policy TD Learning: Finite-Time Analysis with Near-Optimal Sample Complexity and Communication Complexity

The finite-time convergence of off-policy TD learning has been comprehen...

Learning Agent Communication under Limited Bandwidth by Message Pruning

Communication is a crucial factor for the big multi-agent world to stay ...

Connectivity-driven Communication in Multi-agent Reinforcement Learning through Diffusion Processes on Graphs

We discuss the problem of learning collaborative behaviour in multi-agen...

Planning Not to Talk: Multiagent Systems that are Robust to Communication Loss

In a cooperative multiagent system, a collection of agents executes a jo...