CP-MDP: A CANDECOMP-PARAFAC Decomposition Approach to Solve a Markov Decision Process Multidimensional Problem

02/27/2021
by   Daniela Kuinchtner, et al.
0

Markov Decision Process (MDP) is the underlying model for optimal planning for decision-theoretic agents in stochastic environments. Although much research focuses on solving MDP problems both in tabular form or using factored representations, none focused on tensor decomposition methods. Solving MDPs using tensor algebra offers the prospect of leveraging advances in tensor-based computations to further increase solver efficiency. In this paper, we develop an MDP solver for a multidimensional problem using a tensor decomposition method to compress the transition models and optimize the value iteration and policy iteration algorithms. We empirically evaluate our approach against tabular methods and show our approach can compute much larger problems using substantially less memory, opening up new possibilities for tensor-based approaches in stochastic planning

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/01/2019

Elaboration Tolerant Representation of Markov Decision Process via Decision-Theoretic Extension of Probabilistic Action Language pBC+

We extend probabilistic action language pBC+ with the notion of utility ...
05/03/2021

Learning Good State and Action Representations via Tensor Decomposition

The transition kernel of a continuous-state-action Markov decision proce...
04/30/2021

Optimal control policies for resource allocation in the Cloud: comparison between Markov decision process and heuristic approaches

We consider an auto-scaling technique in a cloud system where virtual ma...
10/25/2020

XLVIN: eXecuted Latent Value Iteration Nets

Value Iteration Networks (VINs) have emerged as a popular method to inco...
10/31/2011

Communication-Based Decomposition Mechanisms for Decentralized MDPs

Multi-agent planning in stochastic environments can be framed formally a...
09/26/2019

Markov Decision Process for Video Generation

We identify two pathological cases of temporal inconsistencies in video ...
01/30/2013

Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems

This paper presents two new approaches to decomposing and solving large ...

Code Repositories

cp-mdp

A CANDECOMP-PARAFAC tensor decomposition method to solve a Markov Decision Process (MDP) gridworld problem.


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.