Deterministic Inference of Neural Stochastic Differential Equations

06/16/2020
by   Andreas Look, et al.
0

Model noise is known to have detrimental effects on neural networks, such as training instability and predictive distributions with non-calibrated uncertainty properties. These factors set bottlenecks on the expressive potential of Neural Stochastic Differential Equations (NSDEs), a model family that employs neural nets on both drift and diffusion functions. We introduce a novel algorithm that solves a generic NSDE using only deterministic approximation methods. Given a discretization, we estimate the marginal distribution of the Itô process implied by the NSDE using a recursive scheme to propagate deterministic approximations of the statistical moments across time steps. The proposed algorithm comes with theoretical guarantees on numerical stability and convergence to the true solution, enabling its computational use for robust, accurate, and efficient prediction of long sequences. We observe our novel algorithm to behave interpretably on synthetic setups and to improve the state of the art on two challenging real-world tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/02/2019

Differential Bayesian Neural Nets

Neural Ordinary Differential Equations (N-ODEs) are a powerful building ...
03/19/2021

Multilevel Picard approximations for McKean-Vlasov stochastic differential equations

In the literatur there exist approximation methods for McKean-Vlasov sto...
07/16/2018

Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching

We introduce a novel paradigm for learning non-parametric drift and diff...
03/28/2021

Accurate and Reliable Forecasting using Stochastic Differential Equations

It is critical yet challenging for deep learning models to properly char...
06/17/2020

Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes

We propose a novel scheme for fitting heavily parameterized non-linear s...
06/11/2021

Polynomial propagation of moments in stochastic differential equations

We address the problem of approximating the moments of the solution, X(t...
06/18/2021

Stochastic parareal: an application of probabilistic methods to time-parallelisation

Parareal is a well-studied algorithm for numerically integrating systems...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.