Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programming

02/21/2020
by   Vo Nguyen Le Duy, et al.
0

There is a vast body of literature related to methods for detecting changepoints (CP). However, less attention has been paid to assessing the statistical reliability of the detected CPs. In this paper, we introduce a novel method to perform statistical inference on the significance of the CPs, estimated by a Dynamic Programming (DP)-based optimal CP detection algorithm. Based on the selective inference (SI) framework, we propose an exact (non-asymptotic) approach to compute valid p-values for testing the significance of the CPs. Although it is well-known that SI has low statistical power because of over-conditioning, we address this disadvantage by introducing parametric programming techniques. Then, we propose an efficient method to conduct SI with the minimum amount of conditioning, leading to high statistical power. We conduct experiments on both synthetic and real-world datasets, through which we offer evidence that our proposed method is more powerful than existing methods, has decent performance in terms of computational efficiency, and provides good results in many practical applications.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 13

10/05/2020

Quantifying Statistical Significance of Neural Network Representation-Driven Hypotheses by Selective Inference

In the past few years, various approaches have been developed to explain...
12/25/2020

More Powerful and General Selective Inference for Stepwise Feature Selection using the Homotopy Continuation Approach

Conditional Selective Inference (SI) has been actively studied as a new ...
05/11/2021

More Powerful Conditional Selective Inference for Generalized Lasso by Parametric Programming

Conditional selective inference (SI) has been studied intensively as a n...
10/18/2021

Valid and Exact Statistical Inference for Multi-dimensional Multiple Change-Points by Selective Inference

In this paper, we study statistical inference of change-points (CPs) in ...
04/21/2020

Parametric Programming Approach for Powerful Lasso Selective Inference without Conditioning on Signs

In the past few years, Selective Inference (SI) has been actively studie...
06/03/2019

Computing Valid p-values for Image Segmentation by Selective Inference

Image segmentation is one of the most fundamental tasks of computer visi...
06/01/2017

Selective Inference for Multi-Dimensional Multiple Change Point Detection

We consider the problem of multiple change point (CP) detection from a m...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.