On the Linear Belief Compression of POMDPs: A re-examination of current methods

08/05/2015
by   Zhuoran Wang, et al.
0

Belief compression improves the tractability of large-scale partially observable Markov decision processes (POMDPs) by finding projections from high-dimensional belief space onto low-dimensional approximations, where solving to obtain action selection policies requires fewer computations. This paper develops a unified theoretical framework to analyse three existing linear belief compression approaches, including value-directed compression and two non-negative matrix factorisation (NMF) based algorithms. The results indicate that all the three known belief compression methods have their own critical deficiencies. Therefore, projective NMF belief compression is proposed (P-NMF), aiming to overcome the drawbacks of the existing techniques. The performance of the proposed algorithm is examined on four POMDP problems of reasonably large scale, in comparison with existing techniques. Additionally, the competitiveness of belief compression is compared empirically to a state-of-the-art heuristic search based POMDP solver and their relative merits in solving large-scale POMDPs are investigated.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/30/2011

Finding Approximate POMDP solutions Through Belief Compression

Standard value function approaches to finding policies for Partially Obs...
09/09/2011

Perseus: Randomized Point-based Value Iteration for POMDPs

Partially observable Markov decision processes (POMDPs) form an attracti...
12/06/2017

An Efficient Algorithm for Non-Negative Matrix Factorization with Random Projections

Non-negative matrix factorization (NMF) is one of the most popular decom...
09/30/2011

Anytime Point-Based Approximations for Large POMDPs

The Partially Observable Markov Decision Process has long been recognize...
06/25/2017

There and Back Again: A General Approach to Learning Sparse Models

We propose a simple and efficient approach to learning sparse models. Ou...
11/10/2020

Gaussian Compression Stream: Principle and Preliminary Results

Random projections became popular tools to process big data. In particul...
07/11/2012

Region-Based Incremental Pruning for POMDPs

We present a major improvement to the incremental pruning algorithm for ...