Trust-based Consensus in Multi-Agent Reinforcement Learning Systems

05/25/2022
by   Ho Long Fung, et al.
0

An often neglected issue in multi-agent reinforcement learning (MARL) is the potential presence of unreliable agents in the environment whose deviations from expected behavior can prevent a system from accomplishing its intended tasks. In particular, consensus is a fundamental underpinning problem of cooperative distributed multi-agent systems. Consensus requires different agents, situated in a decentralized communication network, to reach an agreement out of a set of initial proposals that they put forward. Learning-based agents should adopt a protocol that allows them to reach consensus despite having one or more unreliable agents in the system. This paper investigates the problem of unreliable agents in MARL, considering consensus as case study. Echoing established results in the distributed systems literature, our experiments show that even a moderate fraction of such agents can greatly impact the ability of reaching consensus in a networked environment. We propose Reinforcement Learning-based Trusted Consensus (RLTC), a decentralized trust mechanism, in which agents can independently decide which neighbors to communicate with. We empirically demonstrate that our trust mechanism is able to deal with unreliable agents effectively, as evidenced by higher consensus success rates.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/06/2022

Consensus Learning for Cooperative Multi-Agent Reinforcement Learning

Almost all multi-agent reinforcement learning algorithms without communi...
research
10/15/2020

Multi-Agent Trust Region Policy Optimization

We extend trust region policy optimization (TRPO) to multi-agent reinfor...
research
05/09/2023

An Algorithm For Adversary Aware Decentralized Networked MARL

Decentralized multi-agent reinforcement learning (MARL) algorithms have ...
research
04/06/2020

Trust-based Multiagent Consensus or Weightings Aggregation

We introduce a framework for reaching a consensus amongst several agents...
research
11/17/2020

Consensus of Multi-Agent Systems Using Back-Tracking and History Following Algorithms

This paper proposes two algorithms, namely "back-tracking" and "history ...
research
05/10/2021

The Influence of Memory in Multi-Agent Consensus

Multi-agent consensus problems can often be seen as a sequence of autono...
research
11/10/2020

Learning to Communicate and Correct Pose Errors

Learned communication makes multi-agent systems more effective by aggreg...

Please sign up or login with your details

Forgot password? Click here to reset