DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction

04/16/2019
by   Yeqi Liu, et al.
0

Long-term prediction of multivariate time series is still an important but challenging problem. The key to solve this problem is to capture the spatial correlations at the same time, the spatio-temporal relationships at different times and the long-term dependence of the temporal relationships between different series. Attention-based recurrent neural networks (RNN) can effectively represent the dynamic spatio-temporal relationships between exogenous series and target series, but it only performs well in one-step time prediction and short-term time prediction. In this paper, inspired by human attention mechanism including the dual-stage two-phase (DSTP) model and the influence mechanism of target information and non-target information, we propose DSTP-based RNN (DSTP-RNN) and DSTP-RNN-2 respectively for long-term time series prediction. Specifically, we first propose the DSTP-based structure to enhance the spatial correlations between exogenous series. The first phase produces violent but decentralized response weight, while the second phase leads to stationary and concentrated response weight. Secondly, we employ multiple attentions on target series to boost the long-term dependence. Finally, we study the performance of deep spatial attention mechanism and provide experiment and interpretation. Our methods outperform nine baseline methods on four datasets in the fields of energy, finance, environment and medicine, respectively.

READ FULL TEXT
04/07/2017

A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction

The Nonlinear autoregressive exogenous (NARX) model, which predicts the ...
09/13/2021

Prediction of gene expression time series and structural analysis of gene regulatory networks using recurrent neural networks

Methods for time series prediction and classification of gene regulatory...
04/13/2020

Hybrid Attention Networks for Flow and Pressure Forecasting in Water Distribution Systems

Multivariate geo-sensory time series prediction is challenging because o...
03/02/2022

Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Prediction

As industrial systems become more complex and monitoring sensors for eve...
11/09/2018

EA-LSTM: Evolutionary Attention-based LSTM for Time Series Prediction

Time series prediction with deep learning methods, especially long short...
09/11/2018

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

We tackle the long-term prediction of scene evolution in a complex downt...
02/22/2022

Hyper Attention Recurrent Neural Network: Tackling Temporal Covariate Shift in Time Series Analysis

Analyzing long time series with RNNs often suffers from infeasible train...