Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting–Full Version

by   Razvan-Gabriel Cirstea, et al.

A variety of real-world applications rely on far future information to make decisions, thus calling for efficient and accurate long sequence multivariate time series forecasting. While recent attention-based forecasting models show strong abilities in capturing long-term dependencies, they still suffer from two key limitations. First, canonical self attention has a quadratic complexity w.r.t. the input time series length, thus falling short in efficiency. Second, different variables' time series often have distinct temporal dynamics, which existing studies fail to capture, as they use the same model parameter space, e.g., projection matrices, for all variables' time series, thus falling short in accuracy. To ensure high efficiency and accuracy, we propose Triformer, a triangular, variable-specific attention. (i) Linear complexity: we introduce a novel patch attention with linear complexity. When stacking multiple layers of the patch attentions, a triangular structure is proposed such that the layer sizes shrink exponentially, thus maintaining linear complexity. (ii) Variable-specific parameters: we propose a light-weight method to enable distinct sets of model parameters for different variables' time series to enhance accuracy without compromising efficiency and memory usage. Strong empirical evidence on four datasets from multiple domains justifies our design choices, and it demonstrates that Triformer outperforms state-of-the-art methods w.r.t. both accuracy and efficiency. This is an extended version of "Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting", to appear in IJCAI 2022 [Cirstea et al., 2022a], including additional experimental results.


page 1

page 2

page 3

page 4


Towards Spatio-Temporal Aware Traffic Time Series Forecasting–Full Version

Traffic time series forecasting is challenging due to complex spatio-tem...

GCformer: An Efficient Framework for Accurate and Scalable Long-Term Multivariate Time Series Forecasting

Transformer-based models have emerged as promising tools for time series...

Discovering Predictable Latent Factors for Time Series Forecasting

Modern time series forecasting methods, such as Transformer and its vari...

Long-Range Transformers for Dynamic Spatiotemporal Forecasting

Multivariate Time Series Forecasting (TSF) focuses on the prediction of ...

Multi-Variate Time Series Forecasting on Variable Subsets

We formulate a new inference task in the domain of multivariate time ser...

Financial Time Series Representation Learning

This paper addresses the difficulty of forecasting multiple financial ti...

Please sign up or login with your details

Forgot password? Click here to reset