Detecting a common change in multi-stream data or panel data is critical on sequential change-point detection problem. Here, a common change is referred as a change that may occur only in a portion of the N panels; usually caused by external sources. In contrast, the traditional change-point detection is focused on a single sequence (individual panels) where the change is typically caused from internal sources. Several typical detection procedures have been discussed and extended; see Xie and Siegmund (2013), Mei (2013), and Tartakovsky and Veeravalli(2008). Chan (2017) discussed the optimality of detection procedures.
Wu (2019) proposed a combined SR-CUSUM procedure that uses the sum of N Shiryayev-Roberts processes to detect the common change, while the N individual CUSUM processes are used to isolate the changed panels and estimate the change point. The alarming limit is chosen such that the average in-control run length is equal to a designated value. For convenience of discussion, we shall focus on the normal case.
Assume there are independent panels and in panel , the observations follow for and () for if a change occurs at in this panel. Suppose a change may only occur in of the panels, called common change. The
panels can be assumed following a mixture model with probabilityof change in each panel. We shall select the same reference parameter for for all panels.
For , define
as the Shiryayev-Roberts process for the panel and for .
An alarm will be raised at the stopping time
where is chosen such that the is equal to the designated value.
In the normal case, when is small, can be designed by using the following simple approximation (Pollak (1987)):
For example, for , , and corresponding to and , respectively. Further properties on the average run lengths are referred to Wu (2018b). Comparison with other procedures as shown in Wu (2018b) demonstrated that the proposed procedure is very competitive when the proportion is small.
To isolate the changed panels and estimate the common change point, the combined SR-CUSUM procedure calculates the CUSUM processes recursively as
and at the alarm time the change-point for the panel is estimated as the last zero point of ,
for , which is indeed the MLE when . When is unknown, it can be estimated as
Apparently, to isolate the ”true” changed panels, both the change-point estimation (or estimated delay detection time after the change-point estimation) and estimated strength of signals (or ) provide related information. Here we propose a BH-type procedure to control the FDR. In Section 2, we shall first study the corresponding continuous time Brownian motion model and derive the exact null joint distribution for and
for the unchanged panels. The marginal moments and covariance shows that they are highly correlated. Whenis small, we extend the results to the discrete time model in Section 3. In Section 4, we propose to use the approximate null distribution for to form a BH-type procedure to isolate the changed panel by controlling FDR. The isolated changed panels are then used to estimate the common change point. Simulation studies for the FDR, FNR, and biases of estimated common change-point in several typical cases show the proposed method works quite well. The results also help us to select the proper FDR in order to balance FNR.
2 Null Distribution under continuous time model
We assume that a common change is detected (B is large) and the change occurs far away from the beginning. For those unchanged panels, at the detection time, by looking backward at each CUSUM process and using its strong Markov property, we can see that for each , and are approximately equivalent in distribution to the maximum point and the maximum value for a normal random walk for with drift
and variance 1 where
Under the continuous time model, we shall denote for as a Brownian motion and as its corresponding probability measure with drift and as the probability measure when the drift is . Denote
For an independent copy of , we denote . The following theorem gives the joint distribution of and its proof is given in the appendix.
Note that by letting , we see
By taking derivative with respect to x, we get
The following theorem shows that the conditional distribution of given is actually inverse Gaussian under and its proof is given in the appendix.
where and the conditional density function of given is
and the joint density function of is
By integrating with respect to , we have:
Corollary 1. The marginal density function and cdf of are given by
From Theorems 1 and 2, we have the following results and the proofs are given in the appendix.
(i) and ;
(ii) and .
The results show that and are highly correlated. For this reason, we shall consider to isolate the changed panels mainly based on .
3 Approximate null distribution under discrete time model
We derive the Laplace transform for the joint distribution of . Let . Define
and if . It can be seen that
for where .
Thus, we can write
For given , is in distribution equivalent to the sum of k i.i.d. copies of . This leads to the following Laplace transform for in the normal case.
Proof. By conditioning on the value of , we have
From the above theorem , the exact results for moments of and can be obtained. For example,
From the Wiener-Hopf factorization (e.g. Siegmund (1985,Theorem 8.41)) , as the random walk has negative drift, we have
where and .
By letting and , we see
Thus, we have
The following corollary shows the second order approximate exponential property for as .
Corollary 2. As ,
Proof. By taking , we have as ,
As , . Thus, we have
To study the distribution of , we denote by the probability measure with mean and for . We fist note that as ,
For a given large value of , we use Equation of Siegmund (1985) and give the following inverse Gaussian approximation with overshoot correction:
In other words, the unconditional distribution of
can be treated approximately as a mixture of inverse Gaussian distribution.
4 Isolation of change panels and estimation of common change point
Since and are highly correlated, we shall consider the isolation mainly based on
. Conditioning on the common change is detected, we use the corrected exponential distribution forfor for unchanged panels. The BH procedure (Benjamin and Hochberg (1995)) will be used to control the FDR that is defined as the rate of unchanged panels among all claimed changed panels. Similarly, the FNR is defined as the rate of undiscovered true changed panels among all K true changed panels.
We first calculate the p-values by
and be the ordered sequence.
For controlled FDR , the number of isolated changed panels will be defined as
The well-known theoretical results show that the FDR under this procedure has upper bound . Based on the isolated changed panels, we can estimate the common change point based on the corresponding change point estimations for .
To show how the proposed procedure performs, we conduct several simulations and leave theoretical investigation for future consideration.
For , , , (), and the number of changed panels , Table 1 gives the simulation results for FAR (false alarm rate) , FDR, FNR, biases of median estimate and mean estimation based on the change-point estimates from the isolated changed panels, and mean number of total isolated changed panels , along with the conditional average delay detection time (CADT) based on 5000 simulations. All the values are calculated conditioning on the change is detected .
Table 2 gives the corresponding results for () and .
Figure 1 gives the histograms of simulated FDR, FNR, , and for , , , , and conditioning on .
Table 1. Simulation for and with
Table 2. Simulation for and with
By looking at the simulation results of Tables 1 and 2, we have several important findings.
(i) The FDRs are not significantly different between and and decreases when K increases;
(ii) The FNRs are not significantly different when K changes for fixed and decreases when increases;
(iii) The simulated FDRs are very close to the theoretical upper bound ;
(iv) The FNRs decreases as increases and the two are roughly balanced around . So or 0.3 are recommended.
(v) The median estimate for the common change point is preferred as its bias are smaller than the mean estimate;
(vi) The number of isolated panels increases as increases and roughly equals to the true K at .
However as the post-change mean is rarely known and we typically select as the minimum magnitude to detect. So we also run a simulation study for and 2.0. Table 3 gives the results for both and with and 0.3. Additional findings are:
(vii) The FDRs are roughly the same when changes, while the FNRs are reduced more significantly for as increases. From this point of view, is preferred if stronger signals are expected.
(viii) However, as increases, the bias of the common change point becomes more negative, similar to Table 2.1 in Wu (2005, pg 40).
Table 3. Simulation for unknown
In this paper, we proposed a BH procedure to control the FDR after a common change is detected in multi-panel data stream. The method only uses partial information available from each individual CUSUM process and is shown performing quire well. To reduce the FDR for isolating changed panels and estimating the common change point, supplementary runs are necessary on isolated changed panels. A simple method is to run one-sided truncated sequential tests by just finding the true changed panels as discussed in Wu (2018). Further discussions on sequential multiple tests on controlling FDR can also be used in the supplementary runs; see Bartroff (2017), De and Baron (2015), and Song and Fellouris (2019). As discussed in Wu (2019), we may also use the adaptive combined SR-CUSUM procedure which can eliminate large biases of the common change point estimation when the post change means are unknown. The results will be presented in future communications.
6.1 Proof of Theorem 1
where in the second equation from last, we use the fact
The last two terms are evaluated by using Equation (3.14) of Siegmund(1985):
where and are standard normal cdf and pdf.
6.2 Proof of Theorem 2
6.3 Proof of Theorem 3
First, we note and . Second, by using the property of inverse Gaussian distribution,
(ii) is proved by combining the above results.
This research is partially supported by a RSCA grant from California State University at Stanislaus.
- Bartroff, J., 2018.
Multiple hypothesis tests controlling generalized error rates for sequential data. Statistica Sinica 28, 363–398.
- Benjamini,Y., Hochberg,Y.,1995.
Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Roy. Statist. Soc. (B) 57, 289 – 300.
- Chan, H. P.,2017.
Optimal sequential detection in multi-stream data. Annals of Statistics 45(6),2636–2763.
- De, S. K., Baron, M.,2015.
Sequential tests controlling generalized familywise error rates. Statistical Methodology 23,88 – 102.
- Mei, Y.,2010.
Efficient scalable schemes for monitoring a large number of data streams. Biometrika 97,419 –433.
- Pollak, M., 1987.
Average run lengths of an optimal method for detecting a change in distribution. Annals of Statistics 15,749 –779.
- Siegmund, D., 1985.
Sequential Analysis: Tests and Confidence Intervals. New York: Springer.
- Song, Y., Fellouris, G., 2019.
Sequential multiple testing with generalized error control: An asymptotic optimality theory. Ann. Statist 47(3), 1776 -– 1803.
- Tartakovsky, A.G., Veeravalli, V.V., 2008.
Asymptotically optimal quickest detection change detection in distributed sensor. Sequential Analysis 27, 441 –475.
- Wu, Y., 2005.
Inference for Change-point and Post-change Means After a CUSUM Test. Lecture Notes in Statistics 180, Springer, New York .
- Wu, Y., 2018.
Supplementary score test for sparse signals in large-scale truncated sequential tests. Journal of Statistical Theory and Practice 12(4),744–756.
- Wu, Y., 2019.
A combined SR-CUSUM procedure for detecting common changes in panel data. Communication in Statistics: Theory and Methods 48(17): 4302–4319.
- Xie, Y., Siegmund, D.,2013.
Sequential multi-sensor change-point detection. Annals of Statistics 41,670 –692.