
Memoryless Exact Solutions for Deterministic MDPs with Sparse Rewards
We propose an algorithm for deterministic continuous Markov Decision Pro...
read it

Solution Methods for Constrained Markov Decision Process with Continuous Probability Modulation
We propose solution methods for previouslyunsolved constrained MDPs in ...
read it

Efficient Planning in Large MDPs with Weak Linear Function Approximation
Largescale Markov decision processes (MDPs) require planning algorithms...
read it

Effect of Reward Function Choices in MDPs with ValueatRisk
This paper studies ValueatRisk (VaR) problems in short and longhoriz...
read it

Extracting Incentives from BlackBox Decisions
An algorithmic decisionmaker incentivizes people to act in certain ways...
read it

MDPs with Unawareness in Robotics
We formalize decisionmaking problems in robotics and automated control ...
read it

Approximation Methods for Partially Observed Markov Decision Processes (POMDPs)
POMDPs are useful models for systems where the true underlying state is ...
read it
Fast Online Exact Solutions for Deterministic MDPs with Sparse Rewards
Markov Decision Processes (MDPs) are a mathematical framework for modeling sequential decision making under uncertainty. The classical approaches for solving MDPs are well known and have been widely studied, some of which rely on approximation techniques to solve MDPs with large state space and/or action space. However, most of these classical solution approaches and their approximation techniques still take much computation time to converge and usually must be recomputed if the reward function is changed. This paper introduces a novel alternative approach for exactly and efficiently solving deterministic, continuous MDPs with sparse reward sources. When the environment is such that the "distance" between states can be determined in constant time, e.g. grid world, our algorithm offers O( R^2 × A^2 × S), where R is the number of reward sources, A is the number of actions, and S is the number of states. Memory complexity for the algorithm is O( S + R × A). This new approach opens new avenues for boosting computational performance for certain classes of MDPs and is of tremendous value for MDP applications such as robotics and unmanned systems. This paper describes the algorithm and presents numerical experiment results to demonstrate its powerful computational performance. We also provide rigorous mathematical description of the approach.
READ FULL TEXT
Comments
There are no comments yet.