Multi-agent Deep FBSDE Representation For Large Scale Stochastic Differential Games

by   Tianrong Chen, et al.

In this paper, we present a deep learning framework for solving large-scale multi-agent non-cooperative stochastic games using fictitious play. The Hamilton-Jacobi-Bellman (HJB) PDE associated with each agent is reformulated into a set of Forward-Backward Stochastic Differential Equations (FBSDEs) and solved via forward sampling on a suitably defined neural network architecture. Decision-making in multi-agent systems suffers from the curse of dimensionality and strategy degeneration as the number of agents and time horizon increase. We propose a novel Deep FBSDE controller framework which is shown to outperform the current state-of-the-art deep fictitious play algorithm on a high dimensional inter-bank lending/borrowing problem. More importantly, our approach mitigates the curse of many agents and reduces computational and memory complexity, allowing us to scale up to 1,000 agents in simulation, a scale which, to the best of our knowledge, represents a new state of the art. Finally, we showcase the framework's applicability in robotics on a belief-space autonomous racing problem.


page 1

page 2

page 3

page 4


Factorized Q-Learning for Large-Scale Multi-Agent Systems

Deep Q-learning has achieved a significant success in single-agent decis...

Non-cooperative Multi-agent Systems with Exploring Agents

Multi-agent learning is a challenging problem in machine learning that h...

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...

Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control

In this paper, we present a scalable deep learning approach to solve opi...

A coevolutionary approach to deep multi-agent reinforcement learning

Traditionally, Deep Artificial Neural Networks (DNN's) are trained throu...

A Game-Theoretic Account of Responsibility Allocation

When designing or analyzing multi-agent systems, a fundamental problem i...

Continuous Strategy Replicator Dynamics for Multi--Agent Learning

The problem of multi-agent learning and adaptation has attracted a great...