Compact Representation of Value Function in Partially Observable Stochastic Games

03/13/2019
by   Karel Horák, et al.
0

Value methods for solving stochastic games with partial observability model the uncertainty about states of the game as a probability distribution over possible states. The dimension of this belief space is the number of states. For many practical problems, for example in security, there are exponentially many possible states which causes an insufficient scalability of algorithms for real-world problems. To this end, we propose an abstraction technique that addresses this issue of the curse of dimensionality by projecting high-dimensional beliefs to characteristic vectors of significantly lower dimension (e.g., marginal probabilities). Our two main contributions are (1) novel compact representation of the uncertainty in partially observable stochastic games and (2) novel algorithm based on this compact representation that is based on existing state-of-the-art algorithms for solving stochastic games with partial observability. Experimental evaluation confirms that the new algorithm over the compact representation dramatically increases the scalability compared to the state of the art.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/22/2016

Structure in the Value Function of Two-Player Zero-Sum Games of Incomplete Information

Zero-sum stochastic games provide a rich model for competitive decision ...
10/21/2020

Solving Zero-Sum One-Sided Partially Observable Stochastic Games

Many security and other real-world situations are dynamic in nature and ...
09/26/2013

Qualitative Possibilistic Mixed-Observable MDPs

Possibilistic and qualitative POMDPs (pi-POMDPs) are counterparts of POM...
01/10/2013

A Tractable POMDP for a Class of Sequencing Problems

We consider a partially observable Markov decision problem (POMDP) that ...
02/05/2020

Partially Observable Games for Secure Autonomy

Technology development efforts in autonomy and cyber-defense have been e...
06/27/2012

A compact, hierarchical Q-function decomposition

Previous work in hierarchical reinforcement learning has faced a dilemma...
05/09/2012

Temporal Action-Graph Games: A New Representation for Dynamic Games

In this paper we introduce temporal action graph games (TAGGs), a novel ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.