Log In Sign Up

Optimal control of robust team stochastic games

by   Feng Huang, et al.

In stochastic dynamic environments, team stochastic games have emerged as a versatile paradigm for studying sequential decision-making problems of fully cooperative multi-agent systems. However, the optimality of the derived policies is usually sensitive to the model parameters, which are typically unknown and required to be estimated from noisy data in practice. To mitigate the sensitivity of the optimal policy to these uncertain parameters, in this paper, we propose a model of "robust" team stochastic games, where players utilize a robust optimization approach to make decisions. This model extends team stochastic games to the scenario of incomplete information and meanwhile provides an alternative solution concept of robust team optimality. To seek such a solution, we develop a learning algorithm in the form of a Gauss-Seidel modified policy iteration and prove its convergence. This algorithm, compared with robust dynamic programming, not only possesses a faster convergence rate, but also allows for using approximation calculations to alleviate the curse of dimensionality. Moreover, some numerical simulations are presented to demonstrate the effectiveness of the algorithm by generalizing the game model of social dilemmas to sequential robust scenarios.


Logical Team Q-learning: An approach towards factored policies in cooperative MARL

We address the challenge of learning factored policies in cooperative MA...

Characterizing the Decidability of Finite State Automata Team Games with Communication

In this paper we define a new model of limited communication for multipl...

A topology for Team Policies and Existence of Optimal Team Policies in Stochastic Team Theory

In this paper, we establish the existence of team-optimal policies for s...

Limited Resource Optimal Distribution Algorithm Based on Game Iteration Method

The article provides a solution algorithm for the linear programming pro...

Efficiently Computing Nash Equilibria in Adversarial Team Markov Games

Computing Nash equilibrium policies is a central problem in multi-agent ...

Non-signaling Approximations of Stochastic Team Problems

In this paper, we consider non-signaling approximation of finite stochas...