Compact Representation of Value Function in Partially Observable Stochastic Games

by   Karel Horák, et al.

Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our two main contributions are (1) novel compact representation of the uncertainty in partially observable stochastic games and (2) novel algorithm based on this compact representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm over the compact representation dramatically increases the scalability compared to the state of the art.



There are no comments yet.


page 1

page 2

page 3

page 4


Structure in the Value Function of Two-Player Zero-Sum Games of Incomplete Information

Zero-sum stochastic games provide a rich model for competitive decision ...

Solving Zero-Sum One-Sided Partially Observable Stochastic Games

Many security and other real-world situations are dynamic in nature and ...

Qualitative Possibilistic Mixed-Observable MDPs

Possibilistic and qualitative POMDPs (pi-POMDPs) are counterparts of POM...

A Tractable POMDP for a Class of Sequencing Problems

We consider a partially observable Markov decision problem (POMDP) that ...

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been e...

A compact, hierarchical Q-function decomposition

Previous work in hierarchical reinforcement learning has faced a dilemma...

Temporal Action-Graph Games: A New Representation for Dynamic Games

In this paper we introduce temporal action graph games (TAGGs), a novel ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.