Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation

by   Truong-Vinh Hoang, et al.

Filtering is a data assimilation technique that performs the sequential inference of dynamical systems states from noisy observations. Herein, we propose a machine learning-based ensemble conditional mean filter (ML-EnCMF) for tracking possibly high-dimensional non-Gaussian state models with nonlinear dynamics based on sparse observations. The proposed filtering method is developed based on the conditional expectation and numerically implemented using machine learning (ML) techniques combined with the ensemble method. The contribution of this work is twofold. First, we demonstrate that the ensembles assimilated using the ensemble conditional mean filter (EnCMF) provide an unbiased estimator of the Bayesian posterior mean, and their variance matches the expected conditional variance. Second, we implement the EnCMF using artificial neural networks, which have a significant advantage in representing nonlinear functions over high-dimensional domains such as the conditional mean. Finally, we demonstrate the effectiveness of the ML-EnCMF for tracking the states of Lorenz-63 and Lorenz-96 systems under the chaotic regime. Numerical results show that the ML-EnCMF outperforms the ensemble Kalman filter.



There are no comments yet.


page 1

page 2

page 3

page 4


Log-Normalization Constant Estimation using the Ensemble Kalman-Bucy Filter with Application to High-Dimensional Models

In this article we consider the estimation of the log-normalization cons...

Coupling techniques for nonlinear ensemble filtering

We consider filtering in high-dimensional non-Gaussian state-space model...

Data Assimilation by Artificial Neural Networks for an Atmospheric General Circulation Model: Conventional Observation

This paper presents an approach for employing artificial neural networks...

Controlling nonlinear dynamical systems into arbitrary states using machine learning

We propose a novel and fully data driven control scheme which relies on ...

Deep Nonlinear Non-Gaussian Filtering for Dynamical Systems

Filtering is a general name for inferring the states of a dynamical syst...

Auto-differentiable Ensemble Kalman Filters

Data assimilation is concerned with sequentially estimating a temporally...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.