Coach-assisted Multi-Agent Reinforcement Learning Framework for Unexpected Crashed Agents

03/16/2022
by   Jian Zhao, et al.
3

Multi-agent reinforcement learning is difficult to be applied in practice, which is partially due to the gap between the simulated and real-world scenarios. One reason for the gap is that the simulated systems always assume that the agents can work normally all the time, while in practice, one or more agents may unexpectedly "crash" during the coordination process due to inevitable hardware or software failures. Such crashes will destroy the cooperation among agents, leading to performance degradation. In this work, we present a formal formulation of a cooperative multi-agent reinforcement learning system with unexpected crashes. To enhance the robustness of the system to crashes, we propose a coach-assisted multi-agent reinforcement learning framework, which introduces a virtual coach agent to adjust the crash rate during training. We design three coaching strategies and the re-sampling strategy for our coach agent. To the best of our knowledge, this work is the first to study the unexpected crashes in the multi-agent system. Extensive experiments on grid-world and StarCraft II micromanagement tasks demonstrate the efficacy of adaptive strategy compared with the fixed crash rate strategy and curriculum learning strategy. The ablation study further illustrates the effectiveness of our re-sampling strategy.

READ FULL TEXT
research
09/20/2021

Promoting Coordination Through Electing First-moveAgent in Multi-Agent Reinforcement Learning

Learning to coordinate among multiple agents is an essential problem in ...
research
10/10/2021

Cooperative Assistance in Robotic Surgery through Multi-Agent Reinforcement Learning

Cognitive cooperative assistance in robot-assisted surgery holds the pot...
research
02/23/2023

Concept Learning for Interpretable Multi-Agent Reinforcement Learning

Multi-agent robotic systems are increasingly operating in real-world env...
research
04/29/2019

Argus: Smartphone-enabled Human Cooperation via Multi-Agent Reinforcement Learning for Disaster Situational Awareness

Argus exploits a Multi-Agent Reinforcement Learning (MARL) framework to ...
research
05/13/2018

An Optimal Rewiring Strategy for Reinforcement Social Learning in Cooperative Multiagent Systems

Multiagent coordination in cooperative multiagent systems (MASs) has bee...
research
11/05/2019

Learning to flock through reinforcement

Flocks of birds, schools of fish, insects swarms are examples of coordin...
research
12/14/2021

Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module

Designing an effective communication mechanism among agents in reinforce...

Please sign up or login with your details

Forgot password? Click here to reset