ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation

09/14/2021
by   Chuangchuang Sun, et al.
0

In a multirobot system, a number of cyber-physical attacks (e.g., communication hijack, observation perturbations) can challenge the robustness of agents. This robustness issue worsens in multiagent reinforcement learning because there exists the non-stationarity of the environment caused by simultaneously learning agents whose changing policies affect the transition and reward functions. In this paper, we propose a minimax MARL approach to infer the worst-case policy update of other agents. As the minimax formulation is computationally intractable to solve, we apply the convex relaxation of neural networks to solve the inner minimization problem. Such convex relaxation enables robustness in interacting with peer agents that may have significantly different behaviors and also achieves a certified bound of the original optimization problem. We evaluate our approach on multiple mixed cooperative-competitive tasks and show that our method outperforms the previous state of the art approaches on this topic.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

08/05/2020

Robust Deep Reinforcement Learning through Adversarial Loss

Deep neural networks, including reinforcement learning agents, have been...
10/31/2020

A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learning

A fundamental challenge in multiagent reinforcement learning is to learn...
06/16/2021

DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splitting

Analyzing the worst-case performance of deep neural networks against inp...
03/01/2019

TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents

Recent work has identified that classification models implemented as neu...
12/21/2020

Adversarial training for continuous robustness control problem in power systems

We propose a new adversarial training approach for injecting robustness ...
09/18/2019

Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Games

This paper presents an algorithmic framework for learning robust policie...
12/11/2021

Federated Reinforcement Learning at the Edge

Modern cyber-physical architectures use data collected from systems at d...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.