Factorized Q-Learning for Large-Scale Multi-Agent Systems

09/11/2018
by   Yong Chen, et al.
0

Deep Q-learning has achieved a significant success in single-agent decision making tasks. However, it is challenging to extend Q-learning to large-scale multi-agent scenarios, due to the explosion of action space resulting from the complex dynamics between the environment and the agents. In this paper, we propose to make the computation of multi-agent Q-learning tractable by treating the Q-function (w.r.t. state and joint-action) as a high-order high-dimensional tensor and then approximate it with factorized pairwise interactions. Furthermore, we utilize a composite deep neural network architecture for computing the factorized Q-function, share the model parameters among all the agents within the same group, and estimate the agents' optimal joint actions through a coordinate descent type algorithm. All these simplifications greatly reduce the model complexity and accelerate the learning process. Extensive experiments on two different multi-agent problems have demonstrated the performance gain of our proposed approach in comparison with strong baselines, particularly when there are a large number of agents.

READ FULL TEXT

page 6

page 7

page 8

research
11/21/2020

Multi-agent Deep FBSDE Representation For Large Scale Stochastic Differential Games

In this paper, we present a deep learning framework for solving large-sc...
research
07/13/2023

Discovering How Agents Learn Using Few Data

Decentralized learning algorithms are an essential tool for designing mu...
research
05/24/2023

Measuring Causal Responsibility in Multi-Agent Spatial Interactions with Feasible Action-Space Reduction

Modelling causal responsibility in multi-agent spatial interactions is c...
research
05/27/2020

Tensor Decomposition for Multi-agent Predictive State Representation

Predictive state representation (PSR) uses a vector of action-observatio...
research
06/01/2023

EMOTE: An Explainable architecture for Modelling the Other Through Empathy

We can usually assume others have goals analogous to our own. This assum...
research
05/21/2020

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multi-Agent Ensemble Deep Reinforcement Learning with Imitation Acceleration

We consider the optimization of distributed resource scheduling to minim...
research
05/27/2019

CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms

How to optimally dispatch orders to vehicles and how to trade off betwee...

Please sign up or login with your details

Forgot password? Click here to reset