DeepAI
Log In Sign Up

Forecasting of Spatio-temporal Chaotic Dynamics with Recurrent Neural Networks: a comparative study of Reservoir Computing and Backpropagation Algorithms

10/09/2019
by   Pantelis R. Vlachas, et al.
0

How effective are Recurrent Neural Networks (RNNs) in forecasting the spatiotemporal dynamics of chaotic systems ? We address this question through a comparative study of Reservoir Computing (RC) and backpropagation through time (BPTT) algorithms for gated network architectures on a number of benchmark problems. We quantify their relative prediction accuracy on the long-term forecasting of Lorenz-96 and the Kuramoto-Sivashinsky equation and calculation of its Lyapunov spectrum. We discuss their implementation on parallel computers and highlight advantages and limitations of each method. We find that, when the full state dynamics are available for training, RC outperforms BPTT approaches in terms of predictive performance and capturing of the long-term statistics, while at the same time requiring much less time for training. However, in the case of reduced order data, large RC models can be unstable and more likely, than the BPTT algorithms, to diverge in the long term. In contrast, RNNs trained via BPTT capture well the dynamics of these reduced order models. This study confirms that RNNs present a potent computational framework for the forecasting of complex spatio-temporal dynamics.

READ FULL TEXT

page 5

page 13

page 15

page 17

page 19

page 27

page 34

page 37

12/30/2018

Comparison between DeepESNs and gated RNNs on multivariate time-series prediction

We propose an experimental comparison between Deep Echo State Networks (...
11/03/2022

An Improved Time Feedforward Connections Recurrent Neural Networks

Recurrent Neural Networks (RNNs) have been widely applied to deal with t...
06/24/2020

Learning the Effective Dynamics of Complex Multiscale Systems

Simulations of complex multiscale systems are essential for science and ...
11/17/2014

Long-term Recurrent Convolutional Networks for Visual Recognition and Description

Models based on deep convolutional networks have dominated recent image ...
02/21/2020

Convolutional Tensor-Train LSTM for Spatio-temporal Learning

Higher-order Recurrent Neural Networks (RNNs) are effective for long-ter...
03/12/2019

Richness of Deep Echo State Network Dynamics

Reservoir Computing (RC) is a popular methodology for the efficient desi...
11/24/2021

An XGBoost-Based Forecasting Framework for Product Cannibalization

Two major challenges in demand forecasting are product cannibalization a...