Maximum Entropy Inverse Reinforcement Learning for Mean Field Games

04/29/2021
by   Yang Chen, et al.
0

Mean field games (MFG) facilitate the otherwise intractable reinforcement learning (RL) in large-scale multi-agent systems (MAS), through reducing interplays among agents to those between a representative individual agent and the mass of the population. While, RL agents are notoriously prone to unexpected behaviours due to reward mis-specification. This problem is exacerbated by an expanding scale of MAS. Inverse reinforcement learning (IRL) provides a framework to automatically acquire proper reward functions from expert demonstrations. Extending IRL to MFG, however, is challenging due to the complex notion of mean-field-type equilibria and the coupling between agent-level and population-level dynamics. To this end, we propose mean field inverse reinforcement learning (MFIRL), a novel model-free IRL framework for MFG. We derive the algorithm based on a new equilibrium concept that incorporates entropy regularization, and the maximum entropy IRL framework. Experimental results on simulated environments demonstrate that MFIRL is sample efficient and can accurately recover the ground-truth reward functions, compared to the state-of-the-art method.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/30/2019

Multi-Agent Adversarial Inverse Reinforcement Learning

Reinforcement learning agents are prone to undesired behaviors due to re...
02/13/2022

Individual-Level Inverse Reinforcement Learning for Mean Field Games

The recent mean field game (MFG) formalism has enabled the application o...
09/02/2021

Multi-Agent Inverse Reinforcement Learning: Suboptimal Demonstrations and Alternative Solution Concepts

Multi-agent inverse reinforcement learning (MIRL) can be used to learn r...
02/06/2020

Multi Type Mean Field Reinforcement Learning

Mean field theory provides an effective way of scaling multiagent reinfo...
07/08/2021

Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning

Learning in multi-agent systems is highly challenging due to the inheren...
05/17/2021

Mean Field Games Flock! The Reinforcement Learning Way

We present a method enabling a large number of agents to learn how to fl...
02/28/2022

Can Mean Field Control (MFC) Approximate Cooperative Multi Agent Reinforcement Learning (MARL) with Non-Uniform Interaction?

Mean-Field Control (MFC) is a powerful tool to solve Multi-Agent Reinfor...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.