Deterministic Inference of Neural Stochastic Differential Equations

by   Andreas Look, et al.

Model noise is known to have detrimental effects on neural networks, such as training instability and predictive distributions with non-calibrated uncertainty properties. These factors set bottlenecks on the expressive potential of Neural Stochastic Differential Equations (NSDEs), a model family that employs neural nets on both drift and diffusion functions. We introduce a novel algorithm that solves a generic NSDE using only deterministic approximation methods. Given a discretization, we estimate the marginal distribution of the Itô process implied by the NSDE using a recursive scheme to propagate deterministic approximations of the statistical moments across time steps. The proposed algorithm comes with theoretical guarantees on numerical stability and convergence to the true solution, enabling its computational use for robust, accurate, and efficient prediction of long sequences. We observe our novel algorithm to behave interpretably on synthetic setups and to improve the state of the art on two challenging real-world tasks.



There are no comments yet.


page 1

page 2

page 3

page 4


Differential Bayesian Neural Nets

Neural Ordinary Differential Equations (N-ODEs) are a powerful building ...

Multilevel Picard approximations for McKean-Vlasov stochastic differential equations

In the literatur there exist approximation methods for McKean-Vlasov sto...

Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching

We introduce a novel paradigm for learning non-parametric drift and diff...

Accurate and Reliable Forecasting using Stochastic Differential Equations

It is critical yet challenging for deep learning models to properly char...

Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes

We propose a novel scheme for fitting heavily parameterized non-linear s...

Polynomial propagation of moments in stochastic differential equations

We address the problem of approximating the moments of the solution, X(t...

Stochastic parareal: an application of probabilistic methods to time-parallelisation

Parareal is a well-studied algorithm for numerically integrating systems...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.