Value-Decomposition Networks For Cooperative Multi-Agent Learning

06/16/2017
by   Peter Sunehag, et al.
0

We study the problem of cooperative multi-agent reinforcement learning with a single joint reward signal. This class of learning problems is difficult because of the often large combined action and observation spaces. In the fully centralized and decentralized approaches, we find the problem of spurious rewards and a phenomenon we call the "lazy agent" problem, which arises due to partial observability. We address these problems by training individual agents with a novel value decomposition network architecture, which learns to decompose the team value function into agent-wise value functions. We perform an experimental evaluation across a range of partially-observable multi-agent domains and show that learning such value-decompositions leads to superior results, in particular when combined with weight sharing, role information and information channels.

READ FULL TEXT

page 7

page 8

research
03/24/2020

Multi-Agent Reinforcement Learning for Problems with Combined Individual and Team Reward

Many cooperative multi-agent problems require agents to learn individual...
research
04/02/2020

Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) under partial observability ha...
research
08/07/2022

Maximum Correntropy Value Decomposition for Multi-agent Deep Reinforcemen Learning

We explore value decomposition solutions for multi-agent deep reinforcem...
research
05/12/2023

Boosting Value Decomposition via Unit-Wise Attentive State Representation for Cooperative Multi-Agent Reinforcement Learning

In cooperative multi-agent reinforcement learning (MARL), the environmen...
research
09/22/2021

Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning

Cooperative multi-agent reinforcement learning (MARL) faces significant ...
research
04/13/2021

Two-stage training algorithm for AI robot soccer

In multi-agent reinforcement learning, the cooperative learning behavior...
research
06/15/2022

Revisiting Some Common Practices in Cooperative Multi-Agent Reinforcement Learning

Many advances in cooperative multi-agent reinforcement learning (MARL) a...

Please sign up or login with your details

Forgot password? Click here to reset