Partially Observable Online Change Detection via Smooth-Sparse Decomposition

09/22/2020
by   Jie Guo, et al.
12

We consider online change detection of high dimensional data streams with sparse changes, where only a subset of data streams can be observed at each sensing time point due to limited sensing capacities. On the one hand, the detection scheme should be able to deal with partially observable data and meanwhile have efficient detection power for sparse changes. On the other, the scheme should be able to adaptively and actively select the most important variables to observe to maximize the detection power. To address these two points, in this paper, we propose a novel detection scheme called CDSSD. In particular, it describes the structure of high dimensional data with sparse changes by smooth-sparse decomposition, whose parameters can be learned via spike-slab variational Bayesian inference. Then the posterior Bayes factor, which incorporates the learned parameters and sparse change information, is formulated as a detection statistic. Finally, by formulating the statistic as the reward of a combinatorial multi-armed bandit problem, an adaptive sampling strategy based on Thompson sampling is proposed. The efficacy and applicability of our method in practice are demonstrated with numerical studies and a real case study.

READ FULL TEXT

page 27

page 28

page 29

06/05/2019

Robust real-time monitoring of high-dimensional data streams

Robust real-time monitoring of high-dimensional data streams has many im...
05/31/2021

Online Bayesian inference for multiple changepoints and risk assessment

The aim of the present study is to detect abrupt trend changes in the me...
09/24/2020

Bandit Change-Point Detection for Real-Time Monitoring High-Dimensional Data Under Sampling Control

In many real-world problems of real-time monitoring high-dimensional str...
08/06/2019

Online Detection of Sparse Changes in High-Dimensional Data Streams Using Tailored Projections

When applying principal component analysis (PCA) for dimension reduction...
05/25/2022

Scalable Online Change Detection for High-dimensional Data Streams

Detecting changes in data streams is a core objective in their analysis ...
06/05/2019

Detecting linear trend changes and point anomalies in data sequences

We propose TrendSegment, a methodology for detecting multiple change-poi...
09/02/2018

Sequential Detection of Regime Changes in Neural Data

The problem of detecting changes in firing patterns in neural data is st...