Deep Coordination Graphs

09/27/2019
by   Wendelin Böhmer, et al.
54

This paper introduces the deep coordination graph (DCG) for collaborative multi-agent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factorizing the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks and parameter sharing improves generalization over the state-action space. We show that DCG can solve challenging predator-prey tasks that are vulnerable to the relative overgeneralization pathology and in which all other known value factorization approaches fail.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/19/2020

Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) requires coordination to effic...
research
10/11/2019

Learning Nearly Decomposable Value Functions Via Communication Minimization

Reinforcement learning encounters major challenges in multi-agent settin...
research
09/30/2021

Coordinated Reinforcement Learning for Optimizing Mobile Networks

Mobile networks are composed of many base stations and for each of them ...
research
05/25/2022

QGNN: Value Function Factorisation with Graph Neural Networks

In multi-agent reinforcement learning, the use of a global objective is ...
research
12/06/2022

Curriculum Learning for Relative Overgeneralization

In multi-agent reinforcement learning (MARL), many popular methods, such...
research
01/10/2022

Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graph

Existing distributed cooperative multi-agent reinforcement learning (MAR...
research
12/08/2021

Greedy-based Value Representation for Optimal Coordination in Multi-agent Reinforcement Learning

Due to the representation limitation of the joint Q value function, mult...

Please sign up or login with your details

Forgot password? Click here to reset