Preference Communication in Multi-Objective Normal-Form Games

11/17/2021
by   Willem Röpke, et al.
2

We study the problem of multiple agents learning concurrently in a multi-objective environment. Specifically, we consider two agents that repeatedly play a multi-objective normal-form game. In such games, the payoffs resulting from joint actions are vector valued. Taking a utility-based approach, we assume a utility function exists that maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents do not necessarily know their opponent's utility function or strategy, they must learn optimal policies to interact with each other. To aid agents in arriving at adequate solutions, we introduce four novel preference communication protocols for both cooperative as well as self-interested communication. Each approach describes a specific protocol for one agent communicating preferences over their actions and how another agent responds. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can drastically alter the learning process and lead to the emergence of cyclic Nash equilibria which had not been previously observed in this setting. Additionally, we introduce a communication scheme where agents must learn when to communicate. For agents in games with Nash equilibria, we find that communication can be beneficial but difficult to learn when agents have different preferred equilibria. When this is not the case, agents become indifferent to communication. In games without Nash equilibria, our results show differences across learning rates. When using faster learners, we observe that explicit communication becomes more prevalent at around 50 as it helps them in learning a compromise joint policy. Slower learners retain this pattern to a lesser degree, but show increased indifference.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 17

01/17/2020

A utility-based analysis of equilibria in multi-objective normal form games

In multi-objective multi-agent systems (MOMAS), agents explicitly consid...
12/13/2021

On Nash Equilibria in Normal-Form Games With Vectorial Payoffs

We provide an in-depth study of Nash equilibria in multi-objective norma...
11/14/2020

Opponent Learning Awareness and Modelling in Multi-Objective Normal Form Games

Many real-world multi-agent interactions consider multiple distinct crit...
09/13/2018

On Existence, Mixtures, Computation and Efficiency in Multi-objective Games

In a multi-objective game, each individual's payoff is a vector-valued f...
06/13/2021

Decentralized Inertial Best-Response with Voluntary and Limited Communication in Random Communication Networks

Multiple autonomous agents interact over a random communication network ...
03/17/2021

Decentralized Fictitious Play in Near-Potential Games with Time-Varying Communication Networks

We study the convergence properties of decentralized fictitious play (DF...
02/14/2020

Minimax Theorem for Latent Games or: How I Learned to Stop Worrying about Mixed-Nash and Love Neural Nets

Adversarial training, a special case of multi-objective optimization, is...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.