NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting

02/10/2021
by   Kai Chen, et al.
20

Although Transformer has made breakthrough success in widespread domains especially in Natural Language Processing (NLP), applying it to time series forecasting is still a great challenge. In time series forecasting, the autoregressive decoding of canonical Transformer models could introduce huge accumulative errors inevitably. Besides, utilizing Transformer to deal with spatial-temporal dependencies in the problem still faces tough difficulties. To tackle these limitations, this work is the first attempt to propose a Non-Autoregressive Transformer architecture for time series forecasting, aiming at overcoming the time delay and accumulative error issues in the canonical Transformer. Moreover, we present a novel spatial-temporal attention mechanism, building a bridge by a learned temporal influence map to fill the gaps between the spatial and temporal attention, so that spatial and temporal dependencies can be processed integrally. Empirically, we evaluate our model on diversified ego-centric future localization datasets and demonstrate state-of-the-art performance on both real-time and accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/09/2022

Adaptive Graph Spatial-Temporal Transformer Network for Traffic Flow Forecasting

Traffic flow forecasting on graphs has real-world applications in many f...
01/09/2020

Spatial-Temporal Transformer Networks for Traffic Flow Forecasting

Traffic forecasting has emerged as a core component of intelligent trans...
01/18/2022

GTrans: Spatiotemporal Autoregressive Transformer with Graph Embeddings for Nowcasting Extreme Events

Spatiotemporal time series nowcasting should preserve temporal and spati...
06/29/2019

Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting

Time series forecasting is an important problem across many domains, inc...
04/23/2022

Time Series Forecasting (TSF) Using Various Deep Learning Models

Time Series Forecasting (TSF) is used to predict the target variables at...
08/18/2021

Transformers predicting the future. Applying attention in next-frame and time series forecasting

Recurrent Neural Networks were, until recently, one of the best ways to ...
10/13/2021

Benchmarking the Robustness of Spatial-Temporal Models Against Corruptions

The state-of-the-art deep neural networks are vulnerable to common corru...

Code Repositories

Non-AR-Spatial-Temporal-Transformer

Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting.


view repo