Entropy Regularization for Mean Field Games with Learning

09/30/2020
by   Xin Guo, et al.
0

Entropy regularization has been extensively adopted to improve the efficiency, the stability, and the convergence of algorithms in reinforcement learning. This paper analyzes both quantitatively and qualitatively the impact of entropy regularization for Mean Field Game (MFG) with learning in a finite time horizon. Our study provides a theoretical justification that entropy regularization yields time-dependent policies and, furthermore, helps stabilizing and accelerating convergence to the game equilibrium. In addition, this study leads to a policy-gradient algorithm for exploration in MFG. Under this algorithm, agents are able to learn the optimal exploration scheduling, with stable and fast convergence to the game equilibrium.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/24/2020

Q-Learning in Regularized Mean-field Games

In this paper, we introduce a regularized mean-field game and study lear...
research
10/14/2021

Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games

Mean-field games (MFG) were introduced to efficiently analyze approximat...
research
08/16/2020

Global Convergence of Policy Gradient for Linear-Quadratic Mean-Field Control/Game in Continuous Time

Reinforcement learning is a powerful tool to learn the optimal policy of...
research
11/25/2020

Exploratory LQG Mean Field Games with Entropy Regularization

We study a general class of entropy-regularized multi-variate LQG mean f...
research
03/13/2020

A General Framework for Learning Mean-Field Games

This paper presents a general mean-field game (GMFG) framework for simul...
research
02/02/2021

Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning

The recent mean field game (MFG) formalism facilitates otherwise intract...
research
11/27/2018

Understanding the impact of entropy in policy learning

Entropy regularization is commonly used to improve policy optimization i...

Please sign up or login with your details

Forgot password? Click here to reset