Forward-Backward RRT: Branched Sampled FBSDEs for Stochastic Optimal Control

06/22/2020
by   Kelsey P. Hawkins, et al.
0

We propose a numerical method to solve forward-backward stochastic differential equations (FBSDE) arising in stochastic optimal control problems. Instead of sampling forward paths independently, we demonstrate how a rapidly-exploring random tree (RRT) method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. We show how a value function approximation is produced by solving a series of function approximation problems backwards in time along the edges of the constructed RRT tree. We employ a local entropy-weighted least squares Monte Carlo (LSMC) method to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. We demonstrate the proposed method and evaluate it on two nonlinear stochastic optimal control problems with non-quadratic running costs, showing that it can greatly improve convergence over previous FBSDE numerical solution methods.

READ FULL TEXT
research
03/26/2021

On the Time Discretization of the Feynman-Kac Forward-Backward Stochastic Differential Equations for Value Function Approximation

Novel numerical estimators are proposed for the forward-backward stochas...
research
01/31/2022

Imbedding Deep Neural Networks

Continuous depth neural networks, such as Neural ODEs, have refashioned ...
research
03/11/2020

Interpolated Adjoint Method for Neural ODEs

In this paper, we propose a method, which allows us to alleviate or comp...
research
06/25/2023

A Neural RDE approach for continuous-time non-Markovian stochastic control problems

We propose a novel framework for solving continuous-time non-Markovian s...
research
11/24/2020

Reinforced optimal control

Least squares Monte Carlo methods are a popular numerical approximation ...
research
03/04/2021

Differential Flatness as a Sufficient Condition to Generate Optimal Trajectories in Real Time

As robotic systems increase in autonomy, there is a strong need to plan ...
research
01/22/2021

Numerical Methods for Backward Stochastic Differential Equations: A Survey

Backwards Stochastic Differential Equations (BSDEs) have been widely emp...

Please sign up or login with your details

Forgot password? Click here to reset