End-to-End Learning and Intervention in Games

by   Jiayang Li, et al.

In a social system, the self-interest of agents can be detrimental to the collective good, sometimes leading to social dilemmas. To resolve such a conflict, a central designer may intervene by either redesigning the system or incentivizing the agents to change their behaviors. To be effective, the designer must anticipate how the agents react to the intervention, which is dictated by their often unknown payoff functions. Therefore, learning about the agents is a prerequisite for intervention. In this paper, we provide a unified framework for learning and intervention in games. We cast the equilibria of games as individual layers and integrate them into an end-to-end optimization framework. To enable the backward propagation through the equilibria of games, we propose two approaches, respectively based on explicit and implicit differentiation. Specifically, we cast the equilibria as the solutions to variational inequalities (VIs). The explicit approach unrolls the projection method for solving VIs, while the implicit approach exploits the sensitivity of the solutions to VIs. At the core of both approaches is the differentiation through a projection operator. Moreover, we establish the correctness of both approaches and identify the conditions under which one approach is more desirable than the other. The analytical results are validated using several real-world problems.


page 1

page 2

page 3

page 4


Emergent Prosociality in Multi-Agent Games Through Gifting

Coordination is often critical to forming prosocial behaviors – behavior...

On games with coordinating and anti-coordinating agents

This work studies Nash equilibria for heterogeneous games where both coo...

Calibration of Shared Equilibria in General Sum Partially Observable Markov Games

Training multi-agent systems (MAS) to achieve realistic equilibria gives...

Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning

Real economies can be modeled as a sequential imperfect-information game...

Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework

This paper develops a Pontryagin differentiable programming (PDP) method...

Subgame-perfect Equilibria in Mean-payoff Games

In this paper, we provide an effective characterization of all the subga...

Multiagent trajectory models via game theory and implicit layer-based learning

For prediction of interacting agents' trajectories, we propose an end-to...

Please sign up or login with your details

Forgot password? Click here to reset