Fitted Q-Learning in Mean-field Games

12/31/2019
by   Berkay Anahtarcı, et al.
0

In the literature, existence of equilibria for discrete-time mean field games has been in general established via Kakutani's Fixed Point Theorem. However, this fixed point theorem does not entail any iterative scheme for computing equilibria. In this paper, we first propose a Q-iteration algorithm to compute equilibria for mean-field games with known model using Banach Fixed Point Theorem. Then, we generalize this algorithm to model-free setting using fitted Q-iteration algorithm and establish the probabilistic convergence of the proposed iteration. Then, using the output of this learning algorithm, we construct an approximate Nash equilibrium for finite-agent stochastic game with mean-field interaction between agents.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/22/2019

Discrete-time average-cost mean-field games on Polish spaces

In stochastic dynamic games, when the number of players is sufficiently ...
research
02/02/2021

Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning

The recent mean field game (MFG) formalism facilitates otherwise intract...
research
07/04/2019

Approximate Fictitious Play for Mean Field Games

The theory of Mean Field Games (MFG) allows characterizing the Nash equi...
research
06/02/2021

Learn to Predict Equilibria via Fixed Point Networks

Systems of interacting agents can often be modeled as contextual games, ...
research
01/23/2018

A Multiclass Mean-Field Game for Thwarting Misinformation Spread in the Internet of Battlefield Things (IoBT)

In this paper, the problem of misinformation propagation is studied for ...
research
03/13/2020

A General Framework for Learning Mean-Field Games

This paper presents a general mean-field game (GMFG) framework for simul...
research
10/06/2021

Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians

We introduce two algorithms based on a policy iteration method to numeri...

Please sign up or login with your details

Forgot password? Click here to reset