Transformers in Time Series: A Survey

by   Qingsong Wen, et al.

Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also intrigues great interests in the time series community. Among multiple advantages of transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review transformer schemes for time series modeling by highlighting their strengths as well as limitations through a new taxonomy to summarize existing time series transformers in two perspectives. From the perspective of network modifications, we summarize the adaptations of module level and architecture level of the time series transformers. From the perspective of applications, we categorize time series transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. A corresponding resource list that will be continuously updated can be found in the GitHub repository. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.


page 1

page 2

page 3

page 4


W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting

Deep learning utilizing transformers has recently achieved a lot of succ...

ETSformer: Exponential Smoothing Transformers for Time-series Forecasting

Transformers have been actively studied for time-series forecasting in r...

Sparse Binary Transformers for Multivariate Time Series Modeling

Compressed Neural Networks have the potential to enable deep learning ac...

A Survey of Transformers

Transformers have achieved great success in many artificial intelligence...

Transformers in Speech Processing: A Survey

The remarkable success of transformers in the field of natural language ...

Convolution-enhanced Evolving Attention Networks

Attention-based neural networks, such as Transformers, have become ubiqu...

Neural Time Series Analysis with Fourier Transform: A Survey

Recently, Fourier transform has been widely introduced into deep neural ...

Please sign up or login with your details

Forgot password? Click here to reset