Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems

06/21/2022
by   Yanchao Sun, et al.
2

Communication is important in many multi-agent reinforcement learning (MARL) problems for agents to share information and make good decisions. However, when deploying trained communicative agents in a real-world application where noise and potential attackers exist, the safety of communication-based policies becomes a severe issue that is underexplored. Specifically, if communication messages are manipulated by malicious attackers, agents relying on untrustworthy communication may take unsafe actions that lead to catastrophic consequences. Therefore, it is crucial to ensure that agents will not be misled by corrupted communication, while still benefiting from benign communication. In this work, we consider an environment with N agents, where the attacker may arbitrarily change the communication from any C<N-1/2 agents to a victim agent. For this strong threat model, we propose a certifiable defense by constructing a message-ensemble policy that aggregates multiple randomly ablated message sets. Theoretical analysis shows that this message-ensemble policy can utilize benign communication while being certifiably robust to adversarial communication, regardless of the attacking algorithm. Experiments in multiple environments verify that our defense significantly improves the robustness of trained policies against various types of attacks.

READ FULL TEXT

page 9

page 20

research
05/09/2023

Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation

Communication can promote coordination in cooperative Multi-Agent Reinfo...
research
08/09/2021

Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learning

Recent studies in multi-agent communicative reinforcement learning (MACR...
research
10/23/2017

Are Multiagent Systems Resilient to Communication Failures?

A challenge in multiagent control systems is to ensure that they are app...
research
01/17/2022

Planning Not to Talk: Multiagent Systems that are Robust to Communication Loss

In a cooperative multiagent system, a collection of agents executes a jo...
research
05/07/2023

Robust Multi-agent Communication via Multi-view Message Certification

Many multi-agent scenarios require message sharing among agents to promo...
research
05/10/2023

Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers

Cooperative multi-agent reinforcement learning (CMARL) has shown to be p...
research
01/28/2022

Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfire

Malicious agents in collaborative learning and outsourced data collectio...

Please sign up or login with your details

Forgot password? Click here to reset