DeepAI AI Chat
Log In Sign Up

Selective Inference Approach for Statistically Sound Predictive Pattern Mining

02/15/2016
by   Shinya Suzumura, et al.
Nagoya Institute of Technology
Osaka University
The University of Tokyo
0

Discovering statistically significant patterns from databases is an important challenging problem. The main obstacle of this problem is in the difficulty of taking into account the selection bias, i.e., the bias arising from the fact that patterns are selected from extremely large number of candidates in databases. In this paper, we introduce a new approach for predictive pattern mining problems that can address the selection bias issue. Our approach is built on a recently popularized statistical inference framework called selective inference. In selective inference, statistical inferences (such as statistical hypothesis testing) are conducted based on sampling distributions conditional on a selection event. If the selection event is characterized in a tractable way, statistical inferences can be made without minding selection bias issue. However, in pattern mining problems, it is difficult to characterize the entire selection process of mining algorithms. Our main contribution in this paper is to solve this challenging problem for a class of predictive pattern mining problems by introducing a novel algorithmic framework. We demonstrate that our approach is useful for finding statistically significant patterns from databases.

READ FULL TEXT

page 3

page 8

06/26/2015

An Efficient Post-Selection Inference on High-Order Interaction Models

Finding statistically significant high-order interaction features in pre...
08/24/2015

Searching for significant patterns in stratified data

Significant pattern mining, the problem of finding itemsets that are sig...
08/24/2020

Statistically Significant Pattern Mining with Ordinal Utility

Statistically significant patterns mining (SSPM) is an essential and cha...
02/15/2016

Safe Pattern Pruning: An Efficient Approach for Predictive Pattern Mining

In this paper we study predictive pattern mining problems where the goal...
06/09/2021

Fast and More Powerful Selective Inference for Sparse High-order Interaction Model

Automated high-stake decision-making such as medical diagnosis requires ...
06/20/2020

Improving the replicability of results from a single psychological experiment

We identify two aspects of selective inference as major obstacles for re...