Multi-agent Deep FBSDE Representation For Large Scale Stochastic Differential Games

11/21/2020
by   Tianrong Chen, et al.
0

In this paper, we present a deep learning framework for solving large-scale multi-agent non-cooperative stochastic games using fictitious play. The Hamilton-Jacobi-Bellman (HJB) PDE associated with each agent is reformulated into a set of Forward-Backward Stochastic Differential Equations (FBSDEs) and solved via forward sampling on a suitably defined neural network architecture. Decision-making in multi-agent systems suffers from the curse of dimensionality and strategy degeneration as the number of agents and time horizon increase. We propose a novel Deep FBSDE controller framework which is shown to outperform the current state-of-the-art deep fictitious play algorithm on a high dimensional inter-bank lending/borrowing problem. More importantly, our approach mitigates the curse of many agents and reduces computational and memory complexity, allowing us to scale up to 1,000 agents in simulation, a scale which, to the best of our knowledge, represents a new state of the art. Finally, we showcase the framework's applicability in robotics on a belief-space autonomous racing problem.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/11/2018

Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved a significant success in single-agent decis...
05/25/2020

Non-cooperative Multi-agent Systems with Exploring Agents

Multi-agent learning is a challenging problem in machine learning that h...
09/04/2019

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...
04/05/2022

Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control

In this paper, we present a scalable deep learning approach to solve opi...
04/12/2021

A coevolutionary approach to deep multi-agent reinforcement learning

Traditionally, Deep Artificial Neural Networks (DNN's) are trained throu...
05/19/2021

A Game-Theoretic Account of Responsibility Allocation

When designing or analyzing multi-agent systems, a fundamental problem i...
04/29/2009

Continuous Strategy Replicator Dynamics for Multi--Agent Learning

The problem of multi-agent learning and adaptation has attracted a great...