Dynamic Programming for Structured Continuous Markov Decision Problems

07/11/2012
by   Zhengzhu Feng, et al.
0

We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the same throughout the region. We first describe the algorithm for piecewise constant representations. We then extend it to piecewise linear representations, using techniques from POMDPs to represent and reason about linear surfaces efficiently. We show that for complex, structured problems, our approach exploits the natural structure so that optimal solutions can be computed efficiently.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/14/2012

Symbolic Dynamic Programming for Discrete and Continuous State MDPs

Many real-world decision-theoretic planning problems can be naturally mo...
research
07/11/2012

Exploiting First-Order Regression in Inductive Policy Selection

We consider the problem of computing optimal generalised policies for re...
research
01/19/2022

Markov decision processes with observation costs

We present a framework for a controlled Markov chain where the state of ...
research
01/10/2013

A Tractable POMDP for a Class of Sequencing Problems

We consider a partially observable Markov decision problem (POMDP) that ...
research
07/12/2022

Compactly Restrictable Metric Policy Optimization Problems

We study policy optimization problems for deterministic Markov decision ...
research
11/11/2021

Agent Spaces

Exploration is one of the most important tasks in Reinforcement Learning...

Please sign up or login with your details

Forgot password? Click here to reset