Modelling Behavioural Diversity for Learning in Open-Ended Games

03/14/2021
by   Nicolas Perez Nieves, et al.
8

Promoting behavioural diversity is critical for solving games with non-transitive dynamics where strategic cycles exist, and there is no consistent winner (e.g., Rock-Paper-Scissors). Yet, there is a lack of rigorous treatment for defining diversity and constructing diversity-aware learning dynamics. In this work, we offer a geometric interpretation of behavioural diversity in games and introduce a novel diversity metric based on determinantal point processes (DPP). By incorporating the diversity metric into best-response dynamics, we develop diverse fictitious play and diverse policy-space response oracle for solving normal-form games and open-ended games. We prove the uniqueness of the diverse best response and the convergence of our algorithms on two-player games. Importantly, we show that maximising the DPP-based diversity metric guarantees to enlarge the gamescape – convex polytopes spanned by agents' mixtures of strategies. To validate our diversity-aware solvers, we test on tens of games that show strong non-transitivity. Results suggest that our methods achieve much lower exploitability than state-of-the-art solvers by finding effective and diverse strategies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/08/2021

Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity

Strategic diversity is often essential in games: in multi-player games, ...
research
08/24/2023

SC-PSRO: A Unified Strategy Learning Method for Normal-form Games

Solving Nash equilibrium is the key challenge in normal-form games with ...
research
04/20/2020

Approximate exploitability: Learning a best response in large games

A common metric in games of imperfect information is exploitability, i.e...
research
02/25/2019

Pure Strategy Best Responses to Mixed Strategies in Repeated Games

Repeated games are difficult to analyze, especially when agents play mix...
research
03/13/2021

Online Double Oracle

Solving strategic games with huge action space is a critical yet under-e...
research
02/22/2019

From open learners to open games

The categories of open learners (due to Fong, Spivak and Tuyéras) and op...
research
01/05/2023

Evaluating Evasion Strategies in Zebrafish Larvae

An effective evasion strategy allows prey to survive encounters with pre...

Please sign up or login with your details

Forgot password? Click here to reset