What is the best RNN-cell structure for forecasting each time series behavior?

by   Rohaifa Khaldi, et al.

It is unquestionable that time series forecasting is of paramount importance in many fields. The most used machine learning models to address time series forecasting tasks are Recurrent Neural Networks (RNNs). Typically, those models are built using one of the three most popular cells, ELMAN, Long-Short Term Memory (LSTM), or Gated Recurrent Unit (GRU) cells, each cell has a different structure and implies a different computational cost. However, it is not clear why and when to use each RNN-cell structure. Actually, there is no comprehensive characterization of all the possible time series behaviors and no guidance on what RNN cell structure is the most suitable for each behavior. The objective of this study is two-fold: it presents a comprehensive taxonomy of all-time series behaviors (deterministic, random-walk, nonlinear, long-memory, and chaotic), and provides insights into the best RNN cell structure for each time series behavior. We conducted two experiments: (1) The first experiment evaluates and analyzes the role of each component in the LSTM-Vanilla cell by creating 11 variants based on one alteration in its basic architecture (removing, adding, or substituting one cell component). (2) The second experiment evaluates and analyzes the performance of 20 possible RNN-cell structures. Our results showed that the MGU-SLIM3 cell is the most recommended for deterministic and nonlinear behaviors, the MGU-SLIM2 cell is the most suitable for random-walk behavior, FB1 cell is advocated for long-memory behavior, and LSTM-SLIM1 for chaotic behavior.


Recurrent Neural Networks for Forecasting Time Series with Multiple Seasonality: A Comparative Study

This paper compares recurrent neural networks (RNNs) with different type...

ARMA Cell: A Modular and Effective Approach for Neural Autoregressive Modeling

The autoregressive moving average (ARMA) model is a classical, and argua...

Long Short-term Memory RNN

This paper is based on a machine learning project at the Norwegian Unive...

Lessons Learned Applying Deep Learning Approaches to Forecasting Complex Seasonal Behavior

Deep learning methods have gained popularity in recent years through the...

Investigating RNN Memory using Neuro-Evolution: Investigating Recurrent Neural Network Memory Structures using Neuro-Evolution

This paper presents a new algorithm, Evolutionary eXploration of Augment...

Unified recurrent neural network for many feature types

There are time series that are amenable to recurrent neural network (RNN...

A Deep Learning Approach for Forecasting Air Pollution in South Korea Using LSTM

Tackling air pollution is an imperative problem in South Korea, especial...

Please sign up or login with your details

Forgot password? Click here to reset