Learning Trembling Hand Perfect Mean Field Equilibrium for Dynamic Mean Field Games

06/21/2020
by   Kiyeob Lee, et al.
0

Mean Field Games (MFG) are those in which each agent assumes that the states of all others are drawn in an i.i.d. manner from a common belief distribution, and optimizes accordingly. The equilibrium concept here is a Mean Field Equilibrium (MFE), and algorithms for learning MFE in dynamic MFGs are unknown in general due to the non-stationary evolution of the belief distribution. Our focus is on an important subclass that possess a monotonicity property called Strategic Complementarities (MFG-SC). We introduce a natural refinement to the equilibrium concept that we call Trembling-Hand-Perfect MFE (T-MFE), which allows agents to employ a measure of randomization while accounting for the impact of such randomization on their payoffs. We propose a simple algorithm for computing T-MFE under a known model. We introduce both a model-free and a model based approach to learning T-MFE under unknown transition probabilities, using the trembling-hand idea of enabling exploration. We analyze the sample complexity of both algorithms. We also develop a scheme on concurrently sampling the system with a large number of agents that negates the need for a simulator, even though the model is non-stationary. Finally, we empirically evaluate the performance of the proposed algorithms via examples motivated by real-world applications.

READ FULL TEXT

Authors

page 24

03/30/2020

Approximate Equilibrium Computation for Discrete-Time Linear-Quadratic Mean-Field Games

While the topic of mean-field games (MFGs) has a relatively long history...
12/31/2020

Model Free Reinforcement Learning Algorithm for Stationary Mean field Equilibrium for Multiple Types of Agents

We consider a multi-agent Markov strategic interaction over an infinite ...
09/09/2020

Reinforcement Learning in Non-Stationary Discrete-Time Linear-Quadratic Mean-Field Games

In this paper, we study large population multi-agent reinforcement learn...
03/06/2019

Mean Field Equilibrium: Uniqueness, Existence, and Comparative Statics

The standard solution concept for stochastic games is Markov perfect equ...
02/06/2020

Multi Type Mean Field Reinforcement Learning

Mean field theory provides an effective way of scaling multiagent reinfo...
01/16/2020

Sequential decomposition of graphon mean field games

In this paper, we present a sequential decomposition algorithm to comput...
10/12/2018

Mean field voter model on networks and multi-variate beta distribution

The regional characteristics of elections show a strong spatial correlat...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.