Policy Gradient Optimal Correlation Search for Variance Reduction in Monte Carlo simulation and Maximum Optimal Transport

07/24/2023
by   Pierre Bras, et al.
0

We propose a new algorithm for variance reduction when estimating f(X_T) where X is the solution to some stochastic differential equation and f is a test function. The new estimator is (f(X^1_T) + f(X^2_T))/2, where X^1 and X^2 have same marginal law as X but are pathwise correlated so that to reduce the variance. The optimal correlation function ρ is approximated by a deep neural network and is calibrated along the trajectories of (X^1, X^2) by policy gradient and reinforcement learning techniques. Finding an optimal coupling given marginal laws has links with maximum optimal transport.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/08/2019

Trajectory-wise Control Variates for Variance Reduction in Policy Gradient Methods

Policy gradient methods have demonstrated success in reinforcement learn...
research
12/23/2022

An ODE characterisation of multi-marginal optimal transport

The purpose of this paper is to introduce a new numerical method to solv...
research
05/29/2019

Variance Reduction for Evolution Strategies via Structured Control Variates

Evolution Strategies (ES) are a powerful class of blackbox optimization ...
research
02/20/2020

The Directional Optimal Transport

We introduce a constrained optimal transport problem where origins x can...
research
12/20/2022

A Communication Optimal Transport Approach to the Computation of Rate Distortion Functions

In this paper, we propose a new framework named Communication Optimal Tr...
research
12/31/2019

Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation

Sequence generation models are commonly refined with reinforcement learn...
research
06/02/2020

Neural Control Variates

We propose neural control variates (NCV) for unbiased variance reduction...

Please sign up or login with your details

Forgot password? Click here to reset