DeepAI
Log In Sign Up

Auto-differentiable Ensemble Kalman Filters

07/16/2021
by   Yuming Chen, et al.
0

Data assimilation is concerned with sequentially estimating a temporally-evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/06/2020

Online learning of both state and dynamics using ensemble Kalman filters

The reconstruction of the dynamics of an observed physical system as a s...
01/18/2019

Algorithms for high-dimensional non-linear filtering and smoothing problems

Several numerical tools designed to overcome the challenges of smoothing...
07/27/2018

Particle filters for applications in geosciences

Particle filters contain the promise of fully nonlinear data assimilatio...
07/27/2018

Inference of stochastic parameterizations for model error treatment using nested ensemble Kalman filters

Stochastic parameterizations are increasingly being used to represent th...
02/12/2019

Projected Data Assimilation

We introduce a framework for Data Assimilation (DA) in which the data is...