Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems

by   Ashesh Chattopadhyay, et al.
Rice University

Data assimilation (DA) is a key component of many forecasting models in science and engineering. DA allows one to estimate better initial conditions using an imperfect dynamical model of the system and noisy/sparse observations available from the system. Ensemble Kalman filter (EnKF) is a DA algorithm that is widely used in applications involving high-dimensional nonlinear dynamical systems. However, EnKF requires evolving large ensembles of forecasts using the dynamical model of the system. This often becomes computationally intractable, especially when the number of states of the system is very large, e.g., for weather prediction. With small ensembles, the estimated background error covariance matrix in the EnKF algorithm suffers from sampling error, leading to an erroneous estimate of the analysis state (initial condition for the next forecast cycle). In this work, we propose hybrid ensemble Kalman filter (H-EnKF), which is applied to a two-layer quasi-geostrophic flow system as a test case. This framework utilizes a pre-trained deep learning-based data-driven surrogate that inexpensively generates and evolves a large data-driven ensemble of the states of the system to accurately compute the background error covariance matrix with less sampling error. The H-EnKF framework estimates a better initial condition without the need for any ad-hoc localization strategies. H-EnKF can be extended to any ensemble-based DA algorithm, e.g., particle filters, which are currently difficult to use for high dimensional systems.


A multi-model ensemble Kalman filter for data assimilation and forecasting

Data assimilation (DA) aims to optimally combine model forecasts and noi...

Online machine-learning forecast uncertainty estimation for sequential data assimilation

Quantifying forecast uncertainty is a key aspect of state-of-the-art num...

Analysis of a localised nonlinear Ensemble Kalman Bucy Filter with complete and accurate observations

Concurrent observation technologies have made high-precision real-time d...

Integrating Recurrent Neural Networks with Data Assimilation for Scalable Data-Driven State Estimation

Data assimilation (DA) is integrated with machine learning in order to p...

Projected Data Assimilation

We introduce a framework for Data Assimilation (DA) in which the data is...

A Machine Learning Approach to Adaptive Covariance Localization

Data assimilation plays a key role in large-scale atmospheric weather fo...

An elastic framework for ensemble-based large-scale data assimilation

Prediction of chaotic systems relies on a floating fusion of sensor data...

Please sign up or login with your details

Forgot password? Click here to reset