DeepAI AI Chat
Log In Sign Up

For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria

by   Scott Emmons, et al.

Although it has been known since the 1970s that a globally optimal strategy profile in a common-payoff game is a Nash equilibrium, global optimality is a strict requirement that limits the result's applicability. In this work, we show that any locally optimal symmetric strategy profile is also a (global) Nash equilibrium. Furthermore, we show that this result is robust to perturbations to the common payoff and to the local optimum. Applied to machine learning, our result provides a global guarantee for any gradient method that finds a local optimum in symmetric strategy space. While this result indicates stability to unilateral deviation, we nevertheless identify broad classes of games where mixed local optima are unstable under joint, asymmetric deviations. We analyze the prevalence of instability by running learning algorithms in a suite of symmetric games, and we conclude by discussing the applicability of our results to multi-agent RL, cooperative inverse RL, and decentralized POMDPs.


page 2

page 5


Evolution toward a Nash equilibrium

In this paper, we study the dynamic behavior of Hedge, a well-known algo...

Symmetric Decomposition of Asymmetric Games

We introduce new theoretical insights into two-population asymmetric gam...

Nash Equilibria for Exchangeable Team against Team Games, their Mean Field Limit, and Role of Common Randomness

We study stochastic mean-field games among finite number of teams with l...

Decentralized Reinforcement Learning: Global Decision-Making via Local Economic Transactions

This paper seeks to establish a framework for directing a society of sim...

Data Structures for Deviation Payoffs

We present new data structures for representing symmetric normal-form ga...

Optimisation dans la détection de communautés recouvrantes et équilibre de Nash

Community detection in graphs has been the subject of many algorithms. R...

Threshold Tests as Quality Signals: Optimal Strategies, Equilibria, and Price of Anarchy

We study a signaling game between two firms competing to have their prod...