Efficient Importance Sampling via Stochastic Optimal Control for Stochastic Reaction Networks

10/27/2021
by   Chiheb Ben Hammouda, et al.
0

We explore the efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks. We propose a novel importance sampling (IS) approach to improve the efficiency of Monte Carlo (MC) estimators when based on an approximate tau-leap scheme. In the IS framework, it is crucial to choose an appropriate change of probability measure for achieving substantial variance reduction. Based on an original connection between finding the optimal IS parameters within a class of probability measures and a stochastic optimal control (SOC) formulation, we propose an automated approach to obtain a highly efficient path-dependent measure change. The optimal IS parameters are obtained by solving a variance minimization problem. We derive an associated backward equation solved by these optimal parameters. Given the challenge of analytically solving this backward equation, we propose a numerical dynamic programming algorithm to approximate the optimal control parameters. To mitigate the curse of dimensionality issue caused by solving the backward equation in the multi-dimensional case, we propose a learning-based method that approximates the value function using a neural network, the parameters of which are determined via stochastic optimization. Our numerical experiments show that our learning-based IS approach substantially reduces the variance of the MC estimator. Moreover, when applying the numerical dynamic programming approach for the one-dimensional case, we obtained a variance that decays at a rate of 𝒪(Δ t) for a step size of Δ t, compared to 𝒪(1) for a standard MC estimator. For a given prescribed error tolerance, TOL, this implies an improvement in the computational complexity to become 𝒪(TOL^-2) instead of 𝒪(TOL^-3) when using a standard MC estimator.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/05/2023

Automated Importance Sampling via Optimal Control for Stochastic Reaction Networks: A Markovian Projection-based Approach

We propose a novel alternative approach to our previous work (Ben Hammou...
research
07/14/2022

Single Level Importance Sampling for McKean-Vlasov Stochastic Differential Equation

This paper investigates Monte Carlo methods to estimate probabilities of...
research
10/18/2021

Variance Reduction in Stochastic Reaction Networks using Control Variates

Monte Carlo estimation in plays a crucial role in stochastic reaction ne...
research
07/11/2023

Multi-index Importance Sampling for McKean-Vlasov Stochastic Differential Equation

This work introduces a novel approach that combines the multi-index Mont...
research
11/14/2019

Importance sampling for a robust and efficient multilevel Monte Carlo estimator for stochastic reaction networks

The multilevel Monte Carlo (MLMC) method for continuous time Markov chai...
research
11/12/2017

On the Sum of Order Statistics and Applications to Wireless Communication Systems Performances

We consider the problem of evaluating the cumulative distribution functi...

Please sign up or login with your details

Forgot password? Click here to reset