Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress

by   Renjie Wu, et al.

Time series anomaly detection has been a perennially important topic in data science, with papers dating back to the 1950s. However, in recent years there has been an explosion of interest in this topic, much of it driven by the success of deep learning in other domains and for other time series tasks. Most of these papers test on one or more of a handful of popular benchmark datasets, created by Yahoo, Numenta, NASA, etc. In this work we make a surprising claim. The majority of the individual exemplars in these datasets suffer from one or more of four flaws. Because of these four flaws, we believe that many published comparisons of anomaly detection algorithms may be unreliable, and more importantly, much of the apparent progress in recent years may be illusionary. In addition to demonstrating these claims, with this paper we introduce the UCR Time Series Anomaly Datasets. We believe that this resource will perform a similar role as the UCR Time Series Classification Archive, by providing the community with a benchmark that allows meaningful comparisons between approaches and a meaningful gauge of overall progress.



There are no comments yet.


page 1

page 2

page 3

page 4


Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art

Anomaly detection for time-series data has been an important research fi...

Do Deep Neural Networks Contribute to Multivariate Time Series Anomaly Detection?

Anomaly detection in time series is a complex task that has been widely ...

The UCR Time Series Archive

The UCR Time Series Archive - introduced in 2002, has become an importan...

When is Early Classification of Time Series Meaningful?

Since its introduction two decades ago, there has been increasing intere...

FastDTW is approximate and Generally Slower than the Algorithm it Approximates

Many time series data mining problems can be solved with repeated use of...

Time Series Anomaly Detection by Cumulative Radon Features

Detecting anomalous time series is key for scientific, medical and indus...

Time Series Classification to Improve Poultry Welfare

Poultry farms are an important contributor to the human food chain. Worl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.