DeepAI AI Chat
Log In Sign Up

Game-theoretical control with continuous action sets

by   Steven Perkins, et al.

Motivated by the recent applications of game-theoretical learning techniques to the design of distributed control systems, we study a class of control problems that can be formulated as potential games with continuous action sets, and we propose an actor-critic reinforcement learning algorithm that provably converges to equilibrium in this class of problems. The method employed is to analyse the learning process under study through a mean-field dynamical system that evolves in an infinite-dimensional function space (the space of probability distributions over the players' continuous controls). To do so, we extend the theory of finite-dimensional two-timescale stochastic approximation to an infinite-dimensional, Banach space setting, and we prove that the continuous dynamics of the process converge to equilibrium in the case of potential games. These results combine to give a provably-convergent learning algorithm in which players do not need to keep track of the controls selected by the other agents.


page 1

page 2

page 3

page 4


Actor-Critic Provably Finds Nash Equilibria of Linear-Quadratic Mean-Field Games

We study discrete-time mean-field Markov games with infinite numbers of ...

Learning with minimal information in continuous games

We introduce a stochastic learning process called the dampened gradient ...

Q-Learning for Mean-Field Controls

Multi-agent reinforcement learning (MARL) has been applied to many chall...

Reinforcement Learning for Mean Field Games, with Applications to Economics

Mean field games (MFG) and mean field control problems (MFC) are framewo...

Continuous Control for Searching and Planning with a Learned Model

Decision-making agents with planning capabilities have achieved huge suc...

Linear-Quadratic Zero-Sum Mean-Field Type Games: Optimality Conditions and Policy Optimization

In this paper, zero-sum mean-field type games (ZSMFTG) with linear dynam...

A Distributed, Dynamical System View of Finite, Static Games

This paper contains a reformulation of any n-player finite, static game ...