An elastic framework for ensemble-based large-scale data assimilation

11/21/2020
by   Sebastian Friedemann, et al.
0

Prediction of chaotic systems relies on a floating fusion of sensor data (observations) with a numerical model to decide on a good system trajectory and to compensate nonlinear feedback effects. Ensemble-based data assimilation (DA) is a major method for this concern depending on propagating an ensemble of perturbed model realizations.In this paper we develop an elastic, online, fault-tolerant and modular framework called Melissa-DA for large-scale ensemble-based DA. Melissa-DA allows elastic addition or removal of compute resources for state propagation at runtime. Dynamic load balancing based on list scheduling ensuresefficient execution. Online processing of the data produced by ensemble members enables to avoid the I/O bottleneck of file-based approaches. Our implementation embeds the PDAF parallel DA engine, enabling the use of various DA methods. Melissa-DA can support extra ensemble-based DAmethods by implementing the transformation of member background states into analysis states. Experiments confirm the excellent scalability of Melissa-DA, running on up to 16,240 cores, to propagate 16,384 members for a regional hydrological critical zone assimilation relying on theParFlow model on a domain with about 4 M grid cells.

READ FULL TEXT

page 6

page 9

page 11

page 17

page 19

page 22

page 23

page 24

research
05/13/2021

Neuroeconomia e suas interfaces com a Psiquiatria -Saúde mental: Personalidade Consumista uma Patologia Emocional? Prazer ou Necessidade?

Este trabalho apresenta um campo pioneiro no Brasil conhecida como neuro...
research
06/09/2022

Deep learning-enhanced ensemble-based data assimilation for high-dimensional nonlinear dynamical systems

Data assimilation (DA) is a key component of many forecasting models in ...
research
04/02/2016

Online Updating of Word Representations for Part-of-Speech Tagging

We propose online unsupervised domain adaptation (DA), which is performe...
research
09/19/2017

Estimating model evidence using ensemble-based data assimilation with localization - The model selection problem

In recent years, there has been a growing interest in applying data assi...
research
10/29/2020

Domain adaptation under structural causal models

Domain adaptation (DA) arises as an important problem in statistical mac...
research
06/03/2020

Online mirror descent and dual averaging: keeping pace in the dynamic case

Online mirror descent (OMD) and dual averaging (DA) are two fundamental ...

Please sign up or login with your details

Forgot password? Click here to reset