Performance Analysis of Trial and Error Algorithms

11/06/2017
by   Jérôme Gaveau, et al.
0

Model-free decentralized optimizations and learning are receiving increasing attention from theoretical and practical perspectives. In particular, two fully decentralized learning algorithms, namely Trial and Error (TEL) and Optimal Dynamical Learning (ODL), are very appealing for a broad class of games. In fact, ODL has the property to spend a high proportion of time in an optimum state that maximizes the sum of utility of all players. And the TEL has the property to spend a high proportion of time in an optimum state that maximizes the sum of utility of all players if there is a Pure Nash Equilibrium (PNE), otherwise, it spends a high proportion of time in an optimum state that maximizes a tradeoff between the sum of utility of all players and a predefined stability function. On the other hand, estimating the mean fraction of time spent in the optimum state (as well as the mean time duration to reach it) is challenging due to the high complexity and dimension of the inherent Markov Chains. In this paper, under some specific system model, an evaluation of the above performance metrics is provided by proposing an approximation of the considered Markov chains, which allows overcoming the problem of high dimensionality. A comparison between the two algorithms is then performed which allows a better understanding of their performances.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/28/2022

Learning Stationary Nash Equilibrium Policies in n-Player Stochastic Games with Independent Chains via Dual Mirror Descent

We consider a subclass of n-player stochastic games, in which players ha...
research
12/27/2021

Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers?

We study multi-player general-sum Markov games with one of the players d...
research
07/25/2023

Social Optimum Equilibrium Selection for Distributed Multi-Agent Optimization

We study the open question of how players learn to play a social optimum...
research
09/02/2020

Policy Optimization for Linear-Quadratic Zero-Sum Mean-Field Type Games

In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynam...
research
05/29/2022

Independent and Decentralized Learning in Markov Potential Games

We propose a multi-agent reinforcement learning dynamics, and analyze it...
research
11/28/2012

Nature-Inspired Mateheuristic Algorithms: Success and New Challenges

Despite the increasing popularity of metaheuristics, many crucially impo...

Please sign up or login with your details

Forgot password? Click here to reset