DeepAI AI Chat
Log In Sign Up

Game-theoretical control with continuous action sets

12/01/2014
by   Steven Perkins, et al.
0

Motivated by the recent applications of game-theoretical learning techniques to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets, and we propose an actor-critic reinforcement learning algorithm that provably converges to equilibrium in this class of problems. The method employed is to analyse the learning process under study through a mean-field dynamical system that evolves in an infinite-dimensional function space (the space of probability distributions over the players' continuous controls). To do so, we extend the theory of finite-dimensional two-timescale stochastic approximation to an infinite-dimensional, Banach space setting, and we prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provably-convergent learning algorithm in which players do not need to keep track of the controls selected by the other agents.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/16/2019

Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Games

We study discrete-time mean-field Markov games with infinite numbers of ...
06/29/2018

Learning with minimal information in continuous games

We introduce a stochastic learning process called the dampened gradient ...
02/10/2020

Q-Learning for Mean-Field Controls

Multi-agent reinforcement learning (MARL) has been applied to many chall...
06/25/2021

Reinforcement Learning for Mean Field Games, with Applications to Economics

Mean field games (MFG) and mean field control problems (MFC) are framewo...
06/12/2020

Continuous Control for Searching and Planning with a Learned Model

Decision-making agents with planning capabilities have achieved huge suc...
09/01/2020

Linear-Quadratic Zero-Sum Mean-Field Type Games: Optimality Conditions and Policy Optimization

In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynam...
10/09/2017

A Distributed, Dynamical System View of Finite, Static Games

This paper contains a reformulation of any n-player finite, static game ...