Partially Observable Online Change Detection via Smooth-Sparse Decomposition

by   Jie Guo, et al.

We consider online change detection of high dimensional data streams with sparse changes, where only a subset of data streams can be observed at each sensing time point due to limited sensing capacities. On the one hand, the detection scheme should be able to deal with partially observable data and meanwhile have efficient detection power for sparse changes. On the other, the scheme should be able to adaptively and actively select the most important variables to observe to maximize the detection power. To address these two points, in this paper, we propose a novel detection scheme called CDSSD. In particular, it describes the structure of high dimensional data with sparse changes by smooth-sparse decomposition, whose parameters can be learned via spike-slab variational Bayesian inference. Then the posterior Bayes factor, which incorporates the learned parameters and sparse change information, is formulated as a detection statistic. Finally, by formulating the statistic as the reward of a combinatorial multi-armed bandit problem, an adaptive sampling strategy based on Thompson sampling is proposed. The efficacy and applicability of our method in practice are demonstrated with numerical studies and a real case study.


page 27

page 28

page 29


Robust real-time monitoring of high-dimensional data streams

Robust real-time monitoring of high-dimensional data streams has many im...

Online Bayesian inference for multiple changepoints and risk assessment

The aim of the present study is to detect abrupt trend changes in the me...

Bandit Change-Point Detection for Real-Time Monitoring High-Dimensional Data Under Sampling Control

In many real-world problems of real-time monitoring high-dimensional str...

Online Detection of Sparse Changes in High-Dimensional Data Streams Using Tailored Projections

When applying principal component analysis (PCA) for dimension reduction...

Scalable Online Change Detection for High-dimensional Data Streams

Detecting changes in data streams is a core objective in their analysis ...

Detecting linear trend changes and point anomalies in data sequences

We propose TrendSegment, a methodology for detecting multiple change-poi...

Sequential Detection of Regime Changes in Neural Data

The problem of detecting changes in firing patterns in neural data is st...