Accurate Extrinsic Prediction of Physical Systems Using Transformers

by   Arnaud Pannatier, et al.

Accurate high-altitude wind forecasting is important for air traffic control. And the large volume of data available for this task makes deep neural network-based models a possibility. However, special methods are required because the data is measured only sparsely: along the main aircraft trajectories and arranged sparsely in space, namely along the main air corridors. Several deep learning approaches have been proposed, and in this work, we show that Transformers can fit this data efficiently and are able to extrapolate coherently from a context set. We show this by an extensive comparison of Transformers to numerous existing deep learning-based baselines in the literature. Besides high-altitude wind forecasting, we compare competing models on other dynamical physical systems, namely those modelled by partial differential equations, in particular the Poisson equation and Darcy Flow equation. For these experiments, in the case where the data is arranged non-regularly in space, Transformers outperform all the other evaluated methods. We also compared them in a more standard setup where the data is arranged on a grid and show that the Transformers are competitive with state-of-the-art methods, even though it does not require regular spacing. The code and datasets of the different experiments will be made publicly available at publication time.


page 1

page 2

page 3

page 4


Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

In many areas, such as the physical sciences, life sciences, and finance...

A Deep Learning-based Collocation Method for Modeling Unknown PDEs from Sparse Observation

Deep learning-based modeling of dynamical systems driven by partial diff...

Hierarchical Deep Learning of Multiscale Differential Equation Time-Steppers

Nonlinear differential equations rarely admit closed-form solutions, thu...

Solving non-linear Kolmogorov equations in large dimensions by using deep learning: a numerical comparison of discretization schemes

Non-linear partial differential Kolmogorov equations are successfully us...

Meta-Learning-Based Robust Adaptive Flight Control Under Uncertain Wind Conditions

Realtime model learning proves challenging for complex dynamical systems...

DynaBench: A benchmark dataset for learning dynamical systems from low-resolution data

Previous work on learning physical systems from data has focused on high...

Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling

Background: Floods are the most common natural disaster in the world, af...

Please sign up or login with your details

Forgot password? Click here to reset