Solving the Diffusion of Responsibility Problem in Multiagent Reinforcement Learning with a Policy Resonance Approach

08/16/2022
by   Qingxu Fu, et al.
7

SOTA multiagent reinforcement algorithms distinguish themselves in many ways from their single-agent equivalences, except that they still totally inherit the single-agent exploration-exploitation strategy. We report that naively inheriting this strategy from single-agent algorithms causes potential collaboration failures, in which the agents blindly follow mainstream behaviors and reject taking minority responsibility. We named this problem the diffusion of responsibility (DR) as it shares similarities with a same-name social psychology effect. In this work, we start by theoretically analyzing the cause of the DR problem, emphasizing it is not relevant to the reward crafting or the credit assignment problems. We propose a Policy Resonance approach to address the DR problem by modifying the multiagent exploration-exploitation strategy. Next, we show that most SOTA algorithms can equip this approach to promote collaborative agent performance in complex cooperative tasks. Experiments are performed in multiple test benchmark tasks to illustrate the effectiveness of this approach.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 7

page 8

page 10

page 11

research
11/09/2022

Solving Collaborative Dec-POMDPs with Deep Reinforcement Learning Heuristics

WQMIX, QMIX, QTRAN, and VDN are SOTA algorithms for Dec-POMDP. All of th...
research
12/21/2020

Difference Rewards Policy Gradients

Policy gradient methods have become one of the most popular classes of a...
research
05/05/2022

LDSA: Learning Dynamic Subtask Assignment in Cooperative Multi-Agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (MARL) has made prominent...
research
04/21/2019

Generative Exploration and Exploitation

Sparse reward is one of the biggest challenges in reinforcement learning...
research
03/03/2023

Toward Risk-based Optimistic Exploration for Cooperative Multi-Agent Reinforcement Learning

The multi-agent setting is intricate and unpredictable since the behavio...
research
11/03/2022

Theta-Resonance: A Single-Step Reinforcement Learning Method for Design Space Exploration

Given an environment (e.g., a simulator) for evaluating samples in a spe...
research
09/06/2016

Q-Learning with Basic Emotions

Q-learning is a simple and powerful tool in solving dynamic problems whe...

Please sign up or login with your details

Forgot password? Click here to reset