Optimal control of robust team stochastic games

05/16/2021
by   Feng Huang, et al.
0

In stochastic dynamic environments, team stochastic games have emerged as a versatile paradigm for studying sequential decision-making problems of fully cooperative multi-agent systems. However, the optimality of the derived policies is usually sensitive to the model parameters, which are typically unknown and required to be estimated from noisy data in practice. To mitigate the sensitivity of the optimal policy to these uncertain parameters, in this paper, we propose a model of "robust" team stochastic games, where players utilize a robust optimization approach to make decisions. This model extends team stochastic games to the scenario of incomplete information and meanwhile provides an alternative solution concept of robust team optimality. To seek such a solution, we develop a learning algorithm in the form of a Gauss-Seidel modified policy iteration and prove its convergence. This algorithm, compared with robust dynamic programming, not only possesses a faster convergence rate, but also allows for using approximation calculations to alleviate the curse of dimensionality. Moreover, some numerical simulations are presented to demonstrate the effectiveness of the algorithm by generalizing the game model of social dilemmas to sequential robust scenarios.

READ FULL TEXT
research
06/05/2020

Logical Team Q-learning: An approach towards factored policies in cooperative MARL

We address the challenge of learning factored policies in cooperative MA...
research
09/21/2022

Characterizing the Decidability of Finite State Automata Team Games with Communication

In this paper we define a new model of limited communication for multipl...
research
11/02/2017

A topology for Team Policies and Existence of Optimal Team Policies in Stochastic Team Theory

In this paper, we establish the existence of team-optimal policies for s...
research
05/11/2019

Limited Resource Optimal Distribution Algorithm Based on Game Iteration Method

The article provides a solution algorithm for the linear programming pro...
research
08/03/2022

Efficiently Computing Nash Equilibria in Adversarial Team Markov Games

Computing Nash equilibrium policies is a central problem in multi-agent ...
research
05/17/2019

Non-signaling Approximations of Stochastic Team Problems

In this paper, we consider non-signaling approximation of finite stochas...

Please sign up or login with your details

Forgot password? Click here to reset