Statistically Significant Pattern Mining with Ordinal Utility

08/24/2020
by   Thien Q. Tran, et al.
0

Statistically significant patterns mining (SSPM) is an essential and challenging data mining task in the field of knowledge discovery in databases (KDD), in which each pattern is evaluated via a hypothesis test. Our study aims to introduce a preference relation into patterns and to discover the most preferred patterns under the constraint of statistical significance, which has never been considered in existing SSPM problems. We propose an iterative multiple testing procedure that can alternately reject a hypothesis and safely ignore the hypotheses that are less useful than the rejected hypothesis. One advantage of filtering out patterns with low utility is that it avoids consumption of the significance budget by rejection of useless (that is, uninteresting) patterns. This allows the significance budget to be focused on useful patterns, leading to more useful discoveries. We show that the proposed method can control the familywise error rate (FWER) under certain assumptions, that can be satisfied by a realistic problem class in SSPM.We also show that the proposed method always discovers a set of patterns that is at least equally or more useful than those discovered using the standard Tarone-Bonferroni method SSPM.Finally, we conducted several experiments with both synthetic and real-world data to evaluate the performance of our method. As a result, in the experiments with real-world datasets, the proposed method discovered a larger number of more useful patterns than the existing method for all five conducted tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/02/2019

Statistically Significant Discriminative Patterns Searching

Discriminative pattern mining is an essential task of data mining. This ...
research
08/24/2015

Searching for significant patterns in stratified data

Significant pattern mining, the problem of finding itemsets that are sig...
research
02/15/2016

Selective Inference Approach for Statistically Sound Predictive Pattern Mining

Discovering statistically significant patterns from databases is an impo...
research
11/24/2021

Flexible Pattern Discovery and Analysis

Based on the analysis of the proportion of utility in the supporting tra...
research
02/25/2019

Beyond Frequency: Utility Mining with Varied Item-Specific Minimum Utility

Utility-oriented mining which integrates utility theory and data mining ...
research
07/01/2014

Significant Subgraph Mining with Multiple Testing Correction

The problem of finding itemsets that are statistically significantly enr...
research
10/18/2021

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

State-of-the-art deep learning methods achieve human-like performance on...

Please sign up or login with your details

Forgot password? Click here to reset