From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARL

04/28/2023
by   Dapeng Li, et al.
0

Centralized training with decentralized execution (CTDE) is a widely-used learning paradigm that has achieved significant success in complex tasks. However, partial observability issues and the absence of effectively shared signals between agents often limit its effectiveness in fostering cooperation. While communication can address this challenge, it simultaneously reduces the algorithm's practicality. Drawing inspiration from human team cooperative learning, we propose a novel paradigm that facilitates a gradual shift from explicit communication to tacit cooperation. In the initial training stage, we promote cooperation by sharing relevant information among agents and concurrently reconstructing this information using each agent's local trajectory. We then combine the explicitly communicated information with the reconstructed information to obtain mixed information. Throughout the training process, we progressively reduce the proportion of explicitly communicated information, facilitating a seamless transition to fully decentralized execution without communication. Experimental results in various scenarios demonstrate that the performance of our method without communication can approaches or even surpasses that of QMIX and communication-based methods.

READ FULL TEXT

page 4

page 14

research
05/27/2023

Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?

Centralized Training with Decentralized Execution (CTDE) has recently em...
research
10/12/2022

Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learning

We introduce hybrid execution in multi-agent reinforcement learning (MAR...
research
02/14/2023

A Theory of Mind Approach as Test-Time Mitigation Against Emergent Adversarial Communication

Multi-Agent Systems (MAS) is the study of multi-agent interactions in a ...
research
11/28/2021

Evaluating Generalization and Transfer Capacity of Multi-Agent Reinforcement Learning Across Variable Number of Agents

Multi-agent Reinforcement Learning (MARL) problems often require coopera...
research
06/16/2023

Structured Cooperative Learning with Graphical Model Priors

We study how to train personalized models for different tasks on decentr...
research
06/03/2021

Modeling Communication to Coordinate Perspectives in Cooperation

Communication is highly overloaded. Despite this, even young children ar...
research
06/07/2023

Get More for Less in Decentralized Learning Systems

Decentralized learning (DL) systems have been gaining popularity because...

Please sign up or login with your details

Forgot password? Click here to reset