On the Linear Belief Compression of POMDPs: A re-examination of current methods

by   Zhuoran Wang, et al.

Belief compression improves the tractability of large-scale partially observable Markov decision processes (POMDPs) by finding projections from high-dimensional belief space onto low-dimensional approximations, where solving to obtain action selection policies requires fewer computations. This paper develops a unified theoretical framework to analyse three existing linear belief compression approaches, including value-directed compression and two non-negative matrix factorisation (NMF) based algorithms. The results indicate that all the three known belief compression methods have their own critical deficiencies. Therefore, projective NMF belief compression is proposed (P-NMF), aiming to overcome the drawbacks of the existing techniques. The performance of the proposed algorithm is examined on four POMDP problems of reasonably large scale, in comparison with existing techniques. Additionally, the competitiveness of belief compression is compared empirically to a state-of-the-art heuristic search based POMDP solver and their relative merits in solving large-scale POMDPs are investigated.


page 1

page 2

page 3

page 4


Finding Approximate POMDP solutions Through Belief Compression

Standard value function approaches to finding policies for Partially Obs...

Perseus: Randomized Point-based Value Iteration for POMDPs

Partially observable Markov decision processes (POMDPs) form an attracti...

An Efficient Algorithm for Non-Negative Matrix Factorization with Random Projections

Non-negative matrix factorization (NMF) is one of the most popular decom...

Anytime Point-Based Approximations for Large POMDPs

The Partially Observable Markov Decision Process has long been recognize...

There and Back Again: A General Approach to Learning Sparse Models

We propose a simple and efficient approach to learning sparse models. Ou...

Gaussian Compression Stream: Principle and Preliminary Results

Random projections became popular tools to process big data. In particul...

Region-Based Incremental Pruning for POMDPs

We present a major improvement to the incremental pruning algorithm for ...