Learning Graphon Mean Field Games and Approximate Nash Equilibria

11/29/2021
by   Kai Cui, et al.
0

Recent advances at the intersection of dense large graph limits and mean field games have begun to enable the scalable analysis of a broad class of dynamical sequential games with large numbers of agents. So far, results have been largely limited to graphon mean field systems with continuous-time diffusive or jump dynamics, typically without control and with little focus on computational methods. We propose a novel discrete-time formulation for graphon mean field games as the limit of non-linear dense graph Markov games with weak interaction. On the theoretical side, we give extensive and rigorous existence and approximation properties of the graphon mean field solution in sufficiently large systems. On the practical side, we provide general learning schemes for graphon mean field equilibria by either introducing agent equivalence classes or reformulating the graphon mean field system as a classical mean field system. By repeatedly finding a regularized optimal control solution and its generated mean field, we successfully obtain plausible approximate Nash equilibria in otherwise infeasible large dense graph games with many agents. Empirically, we are able to demonstrate on a number of examples that the finite-agent behavior comes increasingly close to the mean field behavior for our computed equilibria as the graph or system size grows, verifying our theory. More generally, we successfully apply policy gradient reinforcement learning in conjunction with sequential Monte Carlo methods.

READ FULL TEXT

page 3

page 30

research
03/30/2022

Hypergraphon Mean Field Games

We propose an approach to modelling large-scale multi-agent dynamical sy...
research
08/31/2022

Recent Advances in Modeling and Control of Epidemics using a Mean Field Approach

Modeling and control of epidemics such as the novel Corona virus have as...
research
09/08/2022

Optimal Offloading Strategies for Edge-Computing via Mean-Field Games and Control

The optimal offloading of tasks in heterogeneous edge-computing scenario...
research
11/16/2021

Learning Equilibria in Mean-Field Games: Introducing Mean-Field PSRO

Recent advances in multiagent learning have seen the introduction ofa fa...
research
07/09/2021

Deep Learning for Mean Field Games and Mean Field Control with Applications to Finance

Financial markets and more generally macro-economic models involve a lar...
research
07/04/2019

A Mean Field Games approach to Cluster Analysis

In this paper, we develop a Mean Field Games approach to Cluster Analysi...
research
07/16/2023

MESOB: Balancing Equilibria Social Optimality

Motivated by bid recommendation in online ad auctions, this paper consid...

Please sign up or login with your details

Forgot password? Click here to reset