Optimistic search strategy: Change point detection for large-scale data via adaptive logarithmic queries

by   Solt Kovács, et al.

As a classical and ever reviving topic, change point detection is often formulated as a search for the maximum of a gain function describing improved fits when segmenting the data. Searching through all candidate split points on the grid for finding the best one requires O(T) evaluations of the gain function for an interval with T observations. If each evaluation is computationally demanding (e.g. in high-dimensional models), this can become infeasible. Instead, we propose optimistic search strategies with O(log T) evaluations exploiting specific structure of the gain function. Towards solid understanding of our strategies, we investigate in detail the classical univariate Gaussian change in mean setup. For some of our proposals we prove asymptotic minimax optimality for single and multiple change point scenarios. Our search strategies generalize far beyond the theoretically analyzed univariate setup. We illustrate, as an example, massive computational speedup in change point detection for high-dimensional Gaussian graphical models. More generally, we demonstrate empirically that our optimistic search methods lead to competitive estimation performance while heavily reducing run-time.



There are no comments yet.


page 1

page 2

page 3

page 4


Seeded Binary Segmentation: A general methodology for fast and optimal change point detection

In recent years, there has been an increasing demand on efficient algori...

A Note on Online Change Point Detection

Online change point detection is originated in sequential analysis, whic...

Adversarially Robust Change Point Detection

Change point detection is becoming increasingly popular in many applicat...

Inference on the Change Point in High Dimensional Dynamic Graphical Models

We propose a new estimator for the change point parameter in a dynamic h...

Change point detection for graphical models in presence of missing values

We propose estimation methods for change points in high-dimensional cova...

Online detection of local abrupt changes in high-dimensional Gaussian graphical models

The problem of identifying change points in high-dimensional Gaussian gr...

A Fast and Efficient Change-point Detection Framework for Modern Data

Change-point analysis is thriving in this big data era to address proble...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.