Toward breaking the curse of dimensionality: an FPTAS for stochastic dynamic programs with multidimensional actions and scalar states

11/28/2018
by   Nir Halman, et al.
0

We propose a Fully Polynomial-Time Approximation Scheme (FPTAS) for stochastic dynamic programs with multidimensional action, scalar state, convex costs and linear state transition function. The action spaces are polyhedral and described by parametric linear programs. This type of problems finds applications in the area of optimal planning under uncertainty, and can be thought of as the problem of optimally managing a single non-discrete resource over a finite time horizon. We show that under a value oracle model for the cost functions this result for one-dimensional state space is "best possible", because a similar dynamic programming model with two-dimensional state space does not admit a PTAS. The FPTAS relies on the solution of polynomial-sized linear programs to recursively compute an approximation of the value function at each stage. Our paper enlarges the class of dynamic programs that admit an FPTAS by showing, under suitable conditions, how to deal with multidimensional action spaces and with vectors of continuous random variables with bounded support. These results bring us one step closer to overcoming the curse of dimensionality of dynamic programming.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/26/2019

Approximate Dynamic Programming with Neural Networks in Linear Discrete Action Spaces

Real-world problems of operations research are typically high-dimensiona...
research
09/10/2019

Multi-Step Greedy and Approximate Real Time Dynamic Programming

Real Time Dynamic Programming (RTDP) is a well-known Dynamic Programming...
research
04/18/2021

Low-rank State-action Value-function Approximation

Value functions are central to Dynamic Programming and Reinforcement Lea...
research
10/15/2020

An Approximate Dynamic Programming Approach to The Incremental Knapsack Problem

We study the incremental knapsack problem, where one wishes to sequentia...
research
06/11/2019

Macro-action Multi-timescale Dynamic Programming for Energy Management with Phase Change Materials

This paper focuses on home energy management systems (HEMS) in buildings...
research
08/05/2020

Semantic verification of dynamic programming

We prove that the generic framework for specifying and solving finite-ho...

Please sign up or login with your details

Forgot password? Click here to reset