A simple efficient density estimator that enables fast systematic search

07/03/2017 ∙ by Jonathan R. Wells, et al. ∙ Federation University Australia 0

This paper introduces a simple and efficient density estimator that enables fast systematic search. To show its advantage over commonly used kernel density estimator, we apply it to outlying aspects mining. Outlying aspects mining discovers feature subsets (or subspaces) that describe how a query stand out from a given dataset. The task demands a systematic search of subspaces. We identify that existing outlying aspects miners are restricted to datasets with small data size and dimensions because they employ kernel density estimator, which is computationally expensive, for subspace assessments. We show that a recent outlying aspects miner can run orders of magnitude faster by simply replacing its density estimator with the proposed density estimator, enabling it to deal with large datasets with thousands of dimensions that would otherwise be impossible.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Kernel density estimator (KDE) (Parzen, 1962; Silverman, 1986) has been the preferred choice for many applications

This paper is under consideration at Pattern Recognition Letters.

. For example, existing outlying aspects miners such as OAMiner (Duan et al., 2015) and Beam (Vinh et al., 2016) have employed KDE to estimate density (as the basis) to assess subspaces.

However, the use of KDE has made both OAMiner and Beam computationally expensive. Thus, they cannot be applied to large datasets with thousands of dimensions.

Grid-based density estimator (Silverman, 1986) is often overlooked because it is considered to be an inferior estimator compared to KDE, though it is more efficient.

Here we show that there is a smoothed version of grid-based density estimator which can produce accurate estimation for outlying aspects mining, and at the same time maintaining the efficiency advantage.

We contribute to

  1. An efficient density estimator which runs orders of magnitude faster than kernel density estimator; yet both estimators produce estimations of similar accuracy for the purpose of outlying aspects mining.

  2. Enabling an existing systematic search method for outlying aspects mining to handle large datasets with thousands of dimensions that would otherwise be infeasible.

  3. Verification of the efficiency and effectiveness of the proposed density estimator using real-world and synthetic datasets. We show that it enables an existing outlying miner to run at least two orders of magnitude faster.

The rest of the paper is organised as follows. Related works are provided in Section 2. The new simple density estimator, which enables existing outlying aspects miners to run orders of magnitude faster, is described in Section 3. The details of employing the density estimator in a recent outlying aspects miner is provided in Section 4. Section 5 explains the time and space complexities. Section 6 reports the empirical evaluation results. Conclusions are provided in the last section.

2 Related works

Kernel density estimator (Silverman, 1986; Scott, 2014) is defined as follows:

(1)

where is a kernel, is the bandwidth, is the query point, and is the given data set.

Both recent outlying aspects miners Beam (Vinh et al., 2016) and OAMiner (Duan et al., 2015) used Härdle’s rule of thumb (Härdle, 1991) (which is based on Silverman’s rule of thumb (Silverman, 1986)) to define the bandwidth of KDE. It is given as follows:

(2)

where

, the interquartile range of the first and third quartiles.

To compute the density of a query in a multi-dimensional subspace , a product kernel (Härdle et al., 2004) is used:

(3)

where is the bandwidth in dimension .

Histogram (Silverman, 1986; Scott, 2014) is also well-studied. One robust way to set the bin width is the Freedman–Diaconis rule (Freedman and Diaconis, 1981):

(4)

where is the interquartile range as defined above.

One common way to get smoothed estimations from histogram is average shifted histogram (Scott, 1985) or its randomised version (Ting et al., 2011; Bourel et al., 2014; Wu et al., 2014), where histograms having different shifted origins are built; and each estimation is an average over the histograms. However, needs to be set to a high number for these methods to work, typically 200 (Bourel et al., 2014) or 1000 (Ting et al., 2011). This kind of estimators will cost times that of a single histogram. Thus, it is not suitable for subspace search applications.

A multi-dimensional histogram is constructed using a multi-dimensional grid, where the bin width in each dimension is set based on one-dimensional histogram.

While histogram is more efficient than KDE, it has larger estimation errors than KDE. Though existing smoothed histograms are competitive, they are not significantly faster than KDE when applied to systematic search. This is the reason why many applications have used KDE. Recent works on outlying aspects mining described below are an example.

OAMiner (Duan et al., 2015) employs a depth-first search that utilises some anti-monotonicity criteria to prune its search space. Given a query and a dataset, it uses a rank score which ranks instances within each subspace based on density of the instances in the subspace. After exploring all the subspaces (bar pruned subspaces), it selects the (minimal) subspaces in which the query is top-ranked, as the outlying aspects of the query with respect to the dataset.

Vinh et al (Vinh et al., 2016)

investigate a suite of score functions, and advocate functions which are dimensionality unbiased, i.e., ‘the average value for any data sample drawn from a uniform distribution is a quantity independent of the dimension of the subspace’. Density is a dimensionality biased score because the density reduced substantially as the number of dimensions increases. They propose to use Z-score as a generic means to convert any dimensionality biased scores to ones which are unbiased. Their evaluation on several scores on some datasets shows that density Z-score is the best. This evaluation is based on a beam search method.

Other existing systematic search methods, such as HiCS (Keller et al., 2012)

, find a list of high contrast subspaces in a preprocessing step, and then apply an anomaly detector to find anomalies for each high contrast subspace. The high contrast score requires a series of statistical tests and a Monte Carlo estimation. It is thus a computationally expensive process, and it needs to be repeated for every candidate subspace, generated from an Apriori-like mechanism as in OAMiner. To improve its runtime, a heuristic pruning rule and an alternative calculation using cumulative entropy must be used

(Böhm et al., 2013). Even with this improvement, it is still computationally more expensive than OAMiner (Duan et al., 2015) and Beam (Vinh et al., 2016).

The above studies in outlying aspect mining have used computationally expensive estimators such as KDE. This constrains the algorithms’ applicability to datasets with low dimensions and small data sizes. We investigate an accurate density estimator based on grid which can expand the algorithms’ applicability to large datasets with high dimensions.

3 sGrid: A Simple Neighbourhood Function for fast density estimation

Let be a -dimensional space of real domain; and be a subspace of dimensions, where .

Given a dataset in , let be a projection in of an instance ; and let be the -subspace neighbourhood of .

To facilitate the estimation of , we employ bins of equal-width111The Freedman–Diaconis rule (Freedman and Diaconis, 1981) mentioned in Section 2 can be used to set the bin width and the bin number for each dimension automatically for a given dataset. for dimension , and the -subspace neighbourhood consists of the bin covering and its neighbouring bins in all directions in the subspace.

Let be the bin which covers and has indices in their respective dimensions in a 2-dimensional space. The neighbouring bins of are . Neighbouring bins in a subspace having dimensions are defined similarly with indices.

Let be the number of bins in . The number of bins used to estimate is if falls in a bin which is surrounded by neighbouring bins in all directions; and when falls in a bin at the edge of any one dimension; and if falls in a bin at the edges of all dimensions (i.e., at one corner of the subspace). Examples of the estimations using neighbouring bins are shown in Figure 1, where is estimated using 9 bins and is estimated using 6 bins because the latter is at the edge on one dimension.

Figure 1: An illustrative example of the new neighbourhood function. The two shaded regions are the neighbouring bins used to estimate density for two instances and .

For all which are close to each other such that , the maximum number of bins that overlap between and is .

In a nutshell, the estimation of is a smoothed version of that provided by an ordinary grid-based estimation method which does not employ neighbouring bins (where ). We name it: sGrid density estimator.

Why does the neighbourhood need to be defined using ? In addition to the smoothing effect, the ordinary grid-based estimation method often creates a situation where contains the same small number of instances in both dense and sparse regions, hampering the capability to find subspaces for an anomaly. Inclusion of neighbouring bins in the estimation enables a better distinction between dense and sparse regions. This is depicted in Figure 1, where either or would have been estimated using one bin, each having 1 instance only, if the ordinary grid-based estimation method is employed. Using the proposed neighbourhood estimation, and are clearly differentiated to be in the dense and sparse regions, respectively, because the former is estimated to have 17 instances and the latter has 1 instance.

Cost of computing . The computation can be done through a bitset intersection operation, i.e., . In other words, it takes intersection operations to get all the instances in the center bin of which has dimensions. As there are bins in , the total time cost is . Using a block size in the bitset operation, the time cost can be reduced to .

In addition, the cost can be significantly reduced by aggregating the three neighbouring bins, i.e., and , in each dimension into one pseudo-bin in preprocessing. A dimension having bins gives rise to pseudo-bins. Instead of spending on bins, can now be computed on pseudo-bins which take only. This is the same time cost for the ordinary grid-based method (or histogram). is used in the experiments reported in Section 6.

4 Using sGrid in outlying aspects miner

To examine the capability of the proposed density estimator, we use a recent outlying aspects miner (Vinh et al., 2016).

We describe the subspace score and the systematic search method used in the following paragraphs.

Density Z-score is identified to be the best score for outlying aspects mining (Vinh et al., 2016). The following definition is based on the density estimator stated in the last section.

Density Z-score. The definition based on density Z-score is given as follows, when density is estimated using :

Definition 1

A query , which stands out from , has its uniqueness described by any one of the subspaces in which is located in a scarcely populated neighbourhood if every subspace in has the Z-score normalised number of instances in the -subspace neighbourhood of less than , i.e., , where is the Z-score222Z-score is used instead of because the latter is a dimensionality biased score, and the former is a dimensionality unbiased score (Vinh et al., 2016). A comparison between subspaces using (or density) is biased towards lower subspaces because as the number of attributes increases, the total space/volume increases exponentially wrt the number of attributes. of ; and

are mean and standard deviation, respectively, of

in the given dataset.

Z-Score standardises any measure. For example, a Z-score value of -2 for an instance means that is two standard deviations smaller than the mean. For anomalies, we are only interested in bins having small number of instances, i.e., having negative Z-score values.

Vinh et al (Vinh et al., 2016) suggest the use of beam search to mine outlying aspects. Beam (Vinh et al., 2016) employs the standard breadth-first search procedure (Russell and Norvig, 2003) that explores a fixed number of subspaces (called beam width) at each level in the search process. A minor adjustment is made to explore all subspaces with two attributes (Vinh et al., 2016). The procedure is reproduced in Algorithm 1 for ease of reference.

Using synthetic datasets, Vinh et al (Vinh et al., 2016) have shown that Beam performed the best using density Z-score. We use this as the basis for comparison.

We produce two versions of Beam, denoted as sGBeam and GBeam. sGBeam employs the proposed sGrid density estimator (where ) to replace KDE to compute the density Z-score in line 10 of Algorithm 1. GBeam uses the ordinary grid-based density estimator (where ). The rest of the procedure is identical.

input :  - given dataset, - query, - maximum search depth, - beam width, - top subspaces as the final output, - set of attributes defining
output :  - Set of subspaces for
1 Initialise ;
2 Generate all subspaces of one & two attributes;
3 Add the top subspace to ;
4 Initialise with top subspaces;
5 for l=3 to  do
6       Initialise ;
7       for each subspace  do
8             for each attribute  do
9                   if {} has not been considered yet then
10                         Compute density Z-score for {};
11                         If the worst scored subspace in is worse than {} then replace;
12                         If then append {}; otherwise, if the worst scored subspace in is worst than {} then replace;
13                        
14                   end if
15                  
16             end for
17            
18       end for
19      
20 end for
21return ;
Algorithm 1 Beam()
Method One estimation estimations
Z-score
Density
Density Z-score
Table 1: Time cost. is the attribute size in a subspace. is the block size in the bitset operation described in Section 3.

5 Time and space complexities

The most expensive part in an outlying aspects miner is the score calculation because it needs to be repeated for every instance in each subspace. The time cost for four different scores are summarised in Table 1.

Note that Z-score is a secondary score derived from a base score such as density or .

Using an efficient method to compute standard deviation (in order to compute density Z-score) takes only; and it only needs to be computed once in each subspace. Therefore, the total time cost of computing density Z-score is for instances, which is repeated for each subspace.

In summary, although density or has the same time complexity, the large constant associated with a density estimator such as KDE makes a huge difference in terms of real compute time. This will be shown in Section 6.1.

The algorithm that uses requires to store instances of attributes and an average of bins per attribute. Thus, it takes space complexity. The algorithm that uses KDE also has space complexity linear to .

6 Empirical Evaluation

The outlying aspects miners used in the comparison are OAMiner (Duan et al., 2015), Beam (Vinh et al., 2016), sGBeam and GBeam. OAMiner and Beam are the latest state-of-the-art outlying aspects miners that use KDE. We use the OAMiner code made available by the authors (Duan et al., 2015). All algorithms are implemented in JAVA333Note that we use the JAVA implementation of Beam that employs KDE in the experiments because the MATLAB implementation provided by the authors (Vinh et al., 2016) produces worse results (see details in Section 6.2).. The only difference between Beam, sGBeam and GBeam, which directly influences their runtimes, is the density estimator used.

The default settings used for these algorithms are: OAMiner (), all beam search methods use beam width = 100 (as used in (Vinh et al., 2016)). The density estimator employed in sGBeam and GBeam uses equal-width bins per dimension; and the Freedman–Diaconis rule (Freedman and Diaconis, 1981) is used to set the bin width 444We have attempted other rules, such as Sturges rule (Sturges, 1926) and Scott’s rule (Scott, 1979), to set the bin width or bin number. The Freedman and Diaconis rule produces the best outlying aspects since it is a more robust rule.. The KDE employed in OAMiner and Beam uses the Gaussian kernel with the default bandwidth (Härdle, 1991), as used by the authors (Duan et al., 2015; Vinh et al., 2016) (stated in Section 2). Euclidean distance was used in both OAMiner and Beam to compute the Gaussian kernel in the experiments.

We conduct two sets of experiments in this section. First, we provide a runtime comparison between OAMiner, Beam, sGBeam and GBeam. The aim is to evaluate their relative runtime performances, as a means to compare the efficiency between the proposed density estimator and KDE used in these algorithms. Second, we evaluate the quality of the subspaces discovered. This examines the quality of the subspaces discovered by these algorithms. Some algorithms produce a long list of subspaces. We examine up to the top 10 subspaces only when assessing the quality of each subspace.

We use six real-world datasets and five synthetic datasets which have diverse data characteristics in the experiments to examine the effects due to large data size and high number of dimensions. The data characteristics are given in Table 2. The synthetic datasets were used by previous studies (Duan et al., 2015; Vinh et al., 2016) and they were created by (Keller et al., 2012)

. Each dataset has some outliers; each outlier stands out from normal clusters in at least 2 to 5 dimensions, and it may be an outlier in multiple subspaces independently. Because these synthetic datasets have ground truths, they are used to assess the quality of the outlying aspects discovered by the miners. NBA was previous used by

(Duan et al., 2015). ALOI is from the MultiView dataset collection (elki.dbs.ifi.lmu.de/wiki/DataSets/MultiView). Other datasets are from the UCI repository (Lichman, 2013).

For each of the real-world datasets, a state-of-the-art anomaly detector called iForest (Liu et al., 2008; Emmott et al., 2013) was used to identify the top ten anomalies; and they were used as queries. Each outlying aspects miner identifies the outlying aspects for each query/anomaly.

All outlying aspects miners conduct the search for subspaces with up to the maximum number of attributes for all datasets. The exceptions are Shuttle and P53Mutant which are used in the scaleup tests; here is used.

Dataset (DS) #Attributes Data size
Synthetic datasets 10 - 50 1,000
Shuttle 10 49,097
NBA 20 220
U2R 34 60,821
ALOI 64 100,000
Har 551 5,272
P53Mutant 5,408 31,159
Table 2: Data characteristics of datasets.

The machine used in the experiments has AMD 16-core CPU running at 2.3GHz with 64GB RAM; and the operating system is Ubuntu 15.04.

6.1 Runtime comparison

The results comparing the runtimes of OAMiner, Beam, sGBeam and GBeam are presented in four groups of datasets: (1) Shuttle and (2) P53Mutant for scaleup tests; (3) four other real-world datasets; and (4) synthetic datasets.

6.1.1 Scaleup test on increasing data sizes

Figure (a)a shows the result on Shuttle. sGBeam and GBeam are the fastest algorithms which have about the same runtimes. OAMiner failed to complete the task when the data size is more than 12000 (25%) because its space complexity is quadratic to data size.

sGBeam and GBeam ran significantly faster than Beam and OAMiner, i.e., two orders of magnitude faster at 25% data size. Note that sGBeam and GBeam at 100% data size ran faster than OAMiner and Beam at 5% data size!

Beam is the slowest but it is able to complete the task using 100% data size, albeit it is three orders of magnitude slower than sGBeam and GBeam.

Note that Beam should run faster than OAMiner because of its reduced search space. But OAMiner pre-computes the pairwise distances and stores them for table lookup in the actual run. The saving in time trades off for an increase in memory requirement. Thus, it limits OAMiner’s applications to small datasets only, as shown in this experiment. OAMiner’s reported runtime excluded the preprocessing time.

6.1.2 Scaleup test on increasing dimensionality

The scaleup test on P53Mutant, as the number of attributes increases from 54 to 5408, is showed in Figure (b)b. Only sGBeam and GBeam were able to complete the scaleup test. Beam could only complete the task with 54 attributes within a reasonable time. Beam is estimated to take more than 100 days to complete the task with 540 attributes; and a number of years for the task with 5408 attributes. OAMiner was unable to run due to out of memory error even with 54 attributes.

[scale=0.5,vecfile=./timing_attribute_size_total-eps-converted-to]./timing_data_size_total

(a) Runtime on Shuttle for 10 queries. Note that OAMiner failed to run using the data size of 50% (24548 data points) on a machine having 64GB of RAM (‘out of memory’ error).

[scale=0.5,vecfile=./timing_data_size_total-eps-converted-to]./timing_attribute_size_total

(b) Runtime on P53Mutant for a single query. OAMiner failed to run due to ‘out of memory’ error. Beam was unable to complete the tasks with higher dimensions within 12 days.
Figure 2: Scaleup tests using : (a) Increasing data size; (b) increasing number of dimensions.
DS OAMiner Beam sGBeam GBeam
NBA 5 116 1.25 1.16
U2R OM 12 Days 4,212 4,170
ALOI OM 12 Days 52,828 47,810
Har OM 12 Days 5,410 4,725
Table 3: Runtime result (seconds) for 10 queries on each real-world dataset. . OM denotes ‘Out of Memory’ error.

6.1.3 Other real-world datasets

Table 3 shows the runtimes on four real-world datasets. Again, only sGbeam and GBeam were able to complete all the tasks. OAMiner had out of memory errors in three large datasets; and Beam could not complete the same three datasets within 12 days. Beam is three orders of magnitude slower than sGbeam and GBeam on the NBA dataset.

DS #Query OAMiner Beam sGBeam GBeam
10D 19 35 986 1.3 1.2
20D 25 1,187 10,242 11 9
30D 44 16,625 26,230 39 24
40D 53 102,457 52,490 75 46
50D 68 451,727 90,075 143 79
Table 4: Runtime results for the synthetic datasets (seconds).

6.1.4 Synthetic datasets

Table 4 shows the runtimes on the five synthetic datasets. sGBeam and GBeam are the fastest algorithms555Note that the differences in runtime between sGBeam and GBeam are due to difference in the total number of subspaces searched for all queries. The algorithm is made efficient as follows: The computed density z-scores for all explored subspaces for the previous queries are saved to be used as a lookup for subsequent queries when the same subspaces need to be explored again. The proposed density estimator used enables sGBeam to explore more distinct subspaces than those explored by GBeam because of sGBeam’s smoothed estimations. For example on 50D, sGBeam explored nearly twice the number of subspaces than GBeam. Despite this, both sGBeam and GBeam have the same order of runtime.; they ran three orders of magnitude faster than Beam, and four orders of magnitude faster than OAMiner on the 50D dataset.

Summary. The runtime comparison shows that the proposed efficient density estimator enables the beam search method to run two to three orders of magnitude faster than that using KDE, enabling the method to deal with large datasets with thousands of dimensions that would otherwise be impossible. The use of KDE slows down OAMiner and Beam significantly in comparison with sGBeam and GBeam.

6.2 Qualitative evaluation

Here we focus on assessing the quality of the subspaces discovered. We use the synthetic datasets, and the three real-world datasets which OAMiner and Beam could run in the last section. They are described in two subsections below.

Synthetic datasets. Table 6 provides the summary results of the five datasets having 10 to 50 dimensions. In each algorithm, we report two qualitative results, i.e., the number of exact matches with the ground truths (shown in parenthesis), and the number of matches which includes subsets or supersets of ground truths, in addition to exact matches.

In terms of the number of exact matches, Beam666Note that the Matlab code of Beam (Vinh et al., 2016) produced worse than the result presented here with an average of 179.5 (98). and sGBeam are the best performers which produced a total of 149.5 and 144 exact matches, respectively. Both results are about 50% more than those obtained by GBeam (97). OAMiner with only 13.5 exact matches is the worst performer.

In terms of the number of matches (which include either subsets or supersets of the ground truths), Beam and GBeam produce approximately the same result (190.5 and 198); followed by sGBeam (179.5) and OAMiner (168).

DS OAMiner Beam sGBeam GBeam
10D 13 (1.5) (19) (19) (14)
20D 22 (1) (10) (10) (10)
30D 37 (3) (38.5) (35.5) (19.5)
40D 44 (4) (32.5) (31) (24)
50D 52 (4) (49.5) (48.5) (29.5)
Total 168 (13.5) (149.5) (144) (97)
Table 6: Examples of matches and exact matches on the 10-dimension synthetic dataset. Discovered subspaces which have exact matches with ground truths are bold-faced. ID is the instance id of a query; GT is the Ground Truth. The numbers in a bracket (i.e., subspace) are attribute indices.
ID GT OAMiner Beam sGBeam GBeam
220 {2,3,4,5} {4} {2} {3} {2,3,4,5} {2,3,4,5} {7,8,6,9} {3,4,5}
315 {0,1} {6,7} {6} {4} {3} {7} {0,1} {0,1} {6,7} {0,1} {6,7} {0,1} {6,7}
323 {8,9} {6} {2,8,9} {8,9} {8,9} {8,9}
577 {2,3,4,5} {0} {4} {6,7} {2,3,4,5} {2,3,4,5} {3,4,5}
723 {2,3,4,5} {0} {3} {9} {2,3,4,5} {2,3,4,5} {2,4}
Table 5: Summary results of five synthetic datasets. Each algorithm shows two results: Number within the parenthesis is the number of exact matches; and the number of matches which includes the subsets and supersets of ground truths in addition to the exact matches. A match with one of the two subspaces that exist in a ground truth is awarded with 0.5.

Table 6 shows examples of the subspaces found by OAMiner, Beam, sGBeam and GBeam on the synthetic dataset having 10 dimensions777Note that the OAMiner result is different from that presented by (Duan et al., 2015) because they have reported to have eliminated all anomalies in the dataset, except the anomaly currently under investigation. Our run includes all anomalies, which is more realistic in practice, as the information of all anomalies is usually unavailable, even with the aid of an anomaly detector. The presented results in both Table 6 and Table 6 were obtained by using the original datasets, rather than the reduced datasets..

OAMiner employs a search pruning rule which relies on minimal subspaces. That has prevented OAMiner from finding the ground truth subspaces which have a high number of attributes, as it stops searching when subspaces with low dimensionality have been found. GBeam has an issue in discovering ground truth subspaces with four or more attributes. This is because the estimated density is not accurate enough. Both sGBeam and Beam have no such issue.

Real-world datasets. Here we focus on Beam and sGBeam to examine how similar their discovered subspaces are. The results from three queries in each of NBA, Shuttle and ALOI888We used 108 points of one cluster only (out of 900 clusters) in ALOI for this experiment. Otherwise Beam could not complete within a reasonable time. are shown in Table 7. The results show that the subspaces discovered by Beam and sGBeam are similar; only a few subspaces found by one algorithm are not in the top ten subspaces discovered by the other algorithm in two queries.

ID Beam sGBeam
0 {3} {2} {19} {1,3} {3} {1,19}
1 {19} {1,19} {6} {1,19} {19} {6}
157 {1,2} {11} {2,3} {1,3,19} {1,2,3} {1,19}
24147 {0,7} {0,4} {0,4,8} {0,4} {0,8} {0,7}
35414 {0,7} {0,4} {0,4,8} {0,4} {0,7} {0,8}
37048 {0,7} {0,4} {0,4,8} {0,4} {0,7} {0,8}
13 {16,36} {0,36} {16,61} {0,16,36} {0,36} {16,36}
24 {0,16} {36} {16,36} {0,16} {36} {0,16,36}
28 {21} {16,61} {16,25} {21} {0,16,61} {0,61}
Table 7: Subspaces discovered by Beam and sGBeam on NBA, Shuttle and ALOI (in the first, second and third rows, respectively). The top three subspaces found for each of three queries are list. indicates the subspace which does not appear in the top ten list discovered by the other algorithm. ID is the instance id of a query; the numbers in a bracket (i.e., subspace) are attribute indices.

Note that we do not have the ground truths to verify the subspaces discovered on the real-world datasets. Without ground truths, there are no suitable summary measures for qualitative assessment of outlying aspects mining. Consensus Index, which is based on entropy, was used previously (Vinh et al., 2016); but it is a measure of consensus of (characteristic) features discovered for instances of a cluster. It is more suitable for measuring clustering results than the ‘correctness’ of distinguishing features of an anomaly, which is the aim of an outlying aspects miner, where different anomalies are likely to have different distinguishing features even with reference to the same cluster.

7 Conclusions

This paper shows that a systematic search method runs orders of magnitude faster by simply replacing the commonly used density estimator with the sGrid density estimator. This enables the systematic search method to handle large datasets with thousands of attributes, an impossible task for existing implementations such as OAMiner (Duan et al., 2015) and Beam (Vinh et al., 2016) that use KDE. We show that sGBeam runs at least two orders of magnitude faster than Beam and OAMiner in our experiments. This is achieved with a minor degradation in terms of the number of exact matches found compared with Beam that uses KDE—a small price to pay for the huge gain in runtime.

Compared with the ordinary grid-based density estimator, sGrid which is a smoothed version overcomes its known weakness to produce more accurate estimations. This enables sGBeam to discover 50% more exact matches than GBeam on the synthetic datasets, while still maintaining the same order of runtime.

sGrid is a generic estimator and can be used in all existing algorithms that employ density estimators. Simply replacing the computationally expensive density estimators with sGrid will significantly improve their runtimes.

References

  • Bourel et al. [2014] M. Bourel, R. Fraiman, and B. Ghattas. Random average shifted histograms. Computational Statistics & Data Analysis, 79:149 – 164, 2014.
  • Böhm et al. [2013] K. Böhm, F. Keller, E. Müller, HV. Nguyen, and J. Vreeken.

    CMI: An information-theoretic contrast measure for enhancing subspace cluster and outlier detection.

    In Proceedings of the 13th SIAM international conference on data mining, pages 198–206, 2013.
  • Duan et al. [2015] Lei Duan, Guanting Tang, Jian Pei, James Bailey, Akiko Campbell, and Changjie Tang. Mining outlying aspects on numeric data. Data Mining and Knowledge Discovery, 29(5):1116–1151, 2015.
  • Emmott et al. [2013] Andrew F. Emmott, Shubhomoy Das, Thomas Dietterich, Alan Fern, and Weng-Keen Wong.

    Systematic construction of anomaly detection benchmarks from real data.

    In Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description, New York, NY, USA, 2013. ACM.
  • Freedman and Diaconis [1981] David Freedman and Persi Diaconis. On the histogram as a density estimator: L2 theory. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 57(4):453–476, 1981.
  • Härdle et al. [2004] W. Härdle, A. Werwatz, M. Müller, and S. Sperlich. Nonparametric and semiparametric models. Springer Series in Statistics, Springer, Berlin, 2004.
  • Härdle [1991] W. Härdle. Smoothing techniques: with implementations in S. Springer, New York, 1991.
  • Keller et al. [2012] F. Keller, E. Muller, and K. Bohm. HiCS: High contrast subspaces for density-based outlier ranking. In Proceedings of IEEE 28th International Conference on Data Engineering, pages 1037–1048, 2012.
  • Lichman [2013] M. Lichman.

    UCI machine learning repository, 2013.

  • Liu et al. [2008] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation Forest. In Proceedings of the 2008 IEEE International Conference on Data Mining, pages 413–422, 2008.
  • Parzen [1962] Emanuel Parzen.

    On estimation of a probability density function and mode.

    33(3):1065–1076, 1962.
  • Russell and Norvig [2003] S. J. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Pearson Education, 2nd edition, 2003.
  • Scott [1979] David W. Scott. On optimal and data-based histograms. Biometrika, 66(3):605, 1979.
  • Scott [1985] David W. Scott. Averaged shifted histograms: Effective nonparametric density estimators in several dimensions. The Annals of Statistics, 13(3):1024–1040, 1985.
  • Scott [2014] David W. Scott. Multivariate density estimation: theory, practice, and visualization. Wiley Series in Probability and Statistics Wiley, New York, 2nd edition, 2014.
  • Silverman [1986] B. Silverman. Density estimation for statistics and data analysis. Chapman & Hall/CRC, 1986.
  • Sturges [1926] Herbert A. Sturges. The choice of a class interval. Journal of the American Statistical Association, 21(153):65–66, 1926.
  • Ting et al. [2011] Kai Ming Ting, Takashi Washio, Jonathan R. Wells, and Fei Tony Liu. Density estimation based on mass. In Proceedings of the 2011 IEEE International Conference on Data Mining, pages 715–724, 2011.
  • Vinh et al. [2016] Nguyen Xuan Vinh, Jeffrey Chan, Simone Romano, James Bailey, Christopher Leckie, Kotagiri Ramamohanarao, and Jian Pei. Discovering outlying aspects in large datasets. Data Mining and Knowledge Discovery, pages 1–36, 2016.
  • Wu et al. [2014] Ke Wu, Kun Zhang, Wei Fan, Andrea Edwards, and Philip S. Yu. RS-Forest: A rapid density estimator for streaming anomaly detection. In Proceedings of the 2014 IEEE International Conference on Data Mining, pages 600–609, 2014.