Maximum Entropy Inverse Reinforcement Learning for Mean Field Games

by   Yang Chen, et al.

Mean field games (MFG) facilitate the otherwise intractable reinforcement learning (RL) in large-scale multi-agent systems (MAS), through reducing interplays among agents to those between a representative individual agent and the mass of the population. While, RL agents are notoriously prone to unexpected behaviours due to reward mis-specification. This problem is exacerbated by an expanding scale of MAS. Inverse reinforcement learning (IRL) provides a framework to automatically acquire proper reward functions from expert demonstrations. Extending IRL to MFG, however, is challenging due to the complex notion of mean-field-type equilibria and the coupling between agent-level and population-level dynamics. To this end, we propose mean field inverse reinforcement learning (MFIRL), a novel model-free IRL framework for MFG. We derive the algorithm based on a new equilibrium concept that incorporates entropy regularization, and the maximum entropy IRL framework. Experimental results on simulated environments demonstrate that MFIRL is sample efficient and can accurately recover the ground-truth reward functions, compared to the state-of-the-art method.



There are no comments yet.


page 1

page 2

page 3

page 4


Multi-Agent Adversarial Inverse Reinforcement Learning

Reinforcement learning agents are prone to undesired behaviors due to re...

Individual-Level Inverse Reinforcement Learning for Mean Field Games

The recent mean field game (MFG) formalism has enabled the application o...

Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts

Multi-agent inverse reinforcement learning (MIRL) can be used to learn r...

Multi Type Mean Field Reinforcement Learning

Mean field theory provides an effective way of scaling multiagent reinfo...

Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning

Learning in multi-agent systems is highly challenging due to the inheren...

Mean Field Games Flock! The Reinforcement Learning Way

We present a method enabling a large number of agents to learn how to fl...

Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?

Mean-Field Control (MFC) is a powerful tool to solve Multi-Agent Reinfor...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.