Approximate Dynamic Programming By Minimizing Distributionally Robust Bounds

05/08/2012
by   Marek Petrik, et al.
0

Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of dimensionality by minimizing a pessimistic bound on the policy loss. This approach turns ADP into an optimization problem, for which we derive new mathematical program formulations and analyze its properties. DRADP improves on the theoretical guarantees of existing ADP methods-it guarantees convergence and L1 norm based error bounds. The empirical evaluation of DRADP shows that the theoretical guarantees translate well into good performance on benchmark problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/20/2022

On the Convergence of Policy Gradient in Robust MDPs

Robust Markov decision processes (RMDPs) are promising models that provi...
research
05/11/2010

Feature Selection Using Regularization in Approximate Linear Programs for Markov Decision Processes

Approximate dynamic programming has been used successfully in a large va...
research
02/27/2023

Optimistic Planning by Regularized Dynamic Programming

We propose a new method for optimistic planning in infinite-horizon disc...
research
04/12/2010

Dynamic Policy Programming

In this paper, we propose a novel policy iteration method, called dynami...
research
07/10/2014

A New Optimal Stepsize For Approximate Dynamic Programming

Approximate dynamic programming (ADP) has proven itself in a wide range ...
research
10/15/2020

An Approximate Dynamic Programming Approach to The Incremental Knapsack Problem

We study the incremental knapsack problem, where one wishes to sequentia...
research
10/22/2019

ALGAMES: A Fast Solver for Constrained Dynamic Games

Dynamic games are an effective paradigm for dealing with the control of ...

Please sign up or login with your details

Forgot password? Click here to reset