DeepAI AI Chat
Log In Sign Up

Compact Representation of Value Function in Partially Observable Stochastic Games

by   Karel Horák, et al.

Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our two main contributions are (1) novel compact representation of the uncertainty in partially observable stochastic games and (2) novel algorithm based on this compact representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm over the compact representation dramatically increases the scalability compared to the state of the art.


page 1

page 2

page 3

page 4


HSVI can solve zero-sum Partially Observable Stochastic Games

State-of-the-art methods for solving 2-player zero-sum imperfect informa...

Structure in the Value Function of Two-Player Zero-Sum Games of Incomplete Information

Zero-sum stochastic games provide a rich model for competitive decision ...

Qualitative Possibilistic Mixed-Observable MDPs

Possibilistic and qualitative POMDPs (pi-POMDPs) are counterparts of POM...

Solving Zero-Sum One-Sided Partially Observable Stochastic Games

Many security and other real-world situations are dynamic in nature and ...

A Tractable POMDP for a Class of Sequencing Problems

We consider a partially observable Markov decision problem (POMDP) that ...

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been e...

HSVI fo zs-POSGs using Concavity, Convexity and Lipschitz Properties

Dynamic programming and heuristic search are at the core of state-of-the...