Log In Sign Up

Auto-differentiable Ensemble Kalman Filters

by   Yuming Chen, et al.

Data assimilation is concerned with sequentially estimating a temporally-evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.


page 1

page 2

page 3

page 4


Online learning of both state and dynamics using ensemble Kalman filters

The reconstruction of the dynamics of an observed physical system as a s...

Algorithms for high-dimensional non-linear filtering and smoothing problems

Several numerical tools designed to overcome the challenges of smoothing...

Particle filters for applications in geosciences

Particle filters contain the promise of fully nonlinear data assimilatio...

Inference of stochastic parameterizations for model error treatment using nested ensemble Kalman filters

Stochastic parameterizations are increasingly being used to represent th...

Projected Data Assimilation

We introduce a framework for Data Assimilation (DA) in which the data is...