An Evaluation of Change Point Detection Algorithms

03/13/2020
by   Gerrit J. J. van den Burg, et al.
0

Change point detection is an important part of time series analysis, as the presence of a change point indicates an abrupt and significant change in the data generating process. While many algorithms for change point detection exist, little attention has been paid to evaluating their performance on real-world time series. Algorithms are typically evaluated on simulated data and a small number of commonly-used series with unreliable ground truth. Clearly this does not provide sufficient insight into the comparative performance of these algorithms. Therefore, instead of developing yet another change point detection method, we consider it vastly more important to properly evaluate existing algorithms on real-world data. To achieve this, we present the first data set specifically designed for the evaluation of change point detection algorithms, consisting of 37 time series from various domains. Each time series was annotated by five expert human annotators to provide ground truth on the presence and location of change points. We analyze the consistency of the human annotators, and describe evaluation metrics that can be used to measure algorithm performance in the presence of multiple ground truth annotations. Subsequently, we present a benchmark study where 13 existing algorithms are evaluated on each of the time series in the data set. This study shows that binary segmentation (Scott and Knott, 1974) and Bayesian online change point detection (Adams and MacKay, 2007) are among the best performing methods. Our aim is that this data set will serve as a proving ground in the development of novel change point detection algorithms.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

01/17/2020

Generalization of Change-Point Detection in Time Series Data Based on Direct Density Ratio Estimation

The goal of the change-point detection is to discover changes of time se...
06/22/2022

SoccerCPD: Formation and Role Change-Point Detection in Soccer Matches Using Spatiotemporal Tracking Data

In fluid team sports such as soccer and basketball, analyzing team forma...
11/28/2020

Time-series Change Point Detection with Self-Supervised Contrastive Predictive Coding

Change Point Detection techniques aim to capture changes in trends and s...
10/28/2019

Harnessing the power of Topological Data Analysis to detect change points in time series

We introduce a novel geometry-oriented methodology, based on the emergin...
03/10/2021

Factorized Binary Search: change point detection in the network structure of multivariate high-dimensional time series

Functional magnetic resonance imaging (fMRI) time series data presents a...
10/26/2017

Segment Parameter Labelling in MCMC Mean-Shift Change Detection

This work addresses the problem of segmentation in time series data with...
03/01/2020

Change Point Detection in Software Performance Testing

We describe our process for automatic detection of performance changes f...

Code Repositories

TCPDBench

The Turing Change Point Detection Benchmark: An Extensive Benchmark Evaluation of Change Point Detection Algorithms on real-world data


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.