High dimensionality: The latest challenge to data analysis

02/12/2019 ∙ by A. M. Pires, et al. ∙ Universidade de Lisboa 0

The advent of modern technology, permitting the measurement of thousands of characteristics simultaneously, has given rise to floods of data characterized by many large or even huge datasets. This new paradigm presents extraordinary challenges to data analysis and the question arises: how can conventional data analysis methods, devised for moderate or small datasets, cope with the complexities of modern data? The case of high dimensional data is particularly revealing of some of the drawbacks. We look at the case where the number of characteristics measured in an object is at least the number of observed objects and conclude that this configuration leads to geometrical and mathematical oddities and is an insurmountable barrier for the direct application of traditional methodologies. If scientists are going to ignore fundamental mathematical results arrived at in this paper and blindly use software to analyze data, the results of their analyses may not be trustful, and the findings of their experiments may never be validated. That is why new methods together with the wise use of traditional approaches are essential to progress safely through the present reality.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

When “objects” (patients, subjects, cells, samples, etc) are measured on distinct, possibly correlated, characteristics (or variables) we say we have a multivariate -dimensional dataset. Examples can be found in all areas of science as well as in many current human activities. Statistical methods to deal with this kind of datasets were developed along the twentieth century under the assumption that the number of variables is much smaller than the number of observations (see e.g., Johnson & Wichern, 2007). However, the automatic acquisition of data due to the extraordinary development of new technologies observed in the last decades, has changed this paradigm. Nowadays it is quite common to have datasets with a number of variables much larger than the number of observations. A variety of examples of such high dimensional datasets can be found in, for instance, the genomics, chemometrics, astronomy, climate or finance literature. How are we dealing with this new scenario? At least two references (Clarke et al., 2008; Johnstone & Titterington, 2009) acknowledge the difficulties encountered and basically recognize our ignorance about the basic properties of high dimensional data spaces. Despite this ignorance a continuous stream of new methods prepared to deal with this kind of data has flooded the scientific literature. For example, chapter 18 of Hastie, Tibshirani & Friedman (2009) is dedicated to high dimensional problems ( much larger than ) and describes many of the methods proposed up to 2008.

In this paper we present some new mathematical results about the geometry of datasets in high dimension which reveal interesting fatal features that have been ignored so far. The consequences of these findings are determinant for the correct approach to analyze data. As a matter of fact it will become clear that it is nonsense to freely use many of the traditional data analysis methods directly to study high dimensional data, in particular if . The paper is organized as follows. In the next section we establish the background, describe the problem and present the main findings. The consequences of the results and possible solutions are discussed in the concluding section.

2 Geometric aspects of multivariate data

2.1 When p is small

In this case is usually much larger than (). If, for instance, the data can be represented by

points in a bivariate scatter plot, where the two axis represent the two variables. The main features of the data (relationship/correlation between the variables, grouping of objects, outliers, and others) can be visualized on this scatter plot. When

things are not so easy. We may produce a 3-d scatter plot, but this scatter plot is ultimately represented in a two dimensional space (computer screen/sheet of paper) and what we really observe is a projection of the 3-d point cloud onto a 2-d subspace. This can be informally described as a picture of the point cloud taken from a given position and angle. To understand a 3-d dataset we must take many pictures from varying positions and angles, which means that visualization of the data is, no doubt, more difficult when than when . What about when ? Conceptually we could always apply a similar procedure, that is, project the p-dimensional data points onto 2-d subspaces. It is easy to understand that, as the number of dimensions increases, we would need an increasingly larger number of pictures to get just a glimpse of the data. This is yet a manifestation of the “curse of dimensionality” Bellman (1957).

At this point we must recognize the need and importance of multivariate statistical methods. These methods are designed to analyze datasets with observations on variables organized in a data matrix, , with rows and columns, where denotes the value of the th variable for the th object. We restrict our attention to numerical variables and we assume throughout that the points are in “general position”, which means that there are no redundancies in the data, like, for instance, two exact replicas of an observation (a precise definition is given in Appendix A).

Multivariate statistical methods (considered in a broad sense, i.e., including machine learning and related topics) can extract and quantify relevant features of the data, and also produce, in many cases, two dimensional graphical representations to complement the analysis. Most of those methods use matrix algebra techniques. For instance, principal components

(Hotelling, 1933)

, which are successive uncorrelated directions with maximal variance. Principal components are defined by the eigenvectors of the covariance matrix,

, or of the correlation matrix,

, whereas the variances along the principal components can be obtained by computing the corresponding eigenvalues. A very important tool in multivariate analysis, which is related to a number of methods, including principal components, is the Mahalanobis distance

(Mahalanobis, 1936).

When comparing/analyzing data on a single variable it is often informative to compute the distance of each observation to the center of the data (usually identified by the sample mean) or the distance between pairs of observations, taking into consideration the intrinsic variability of the data, usually measured by the standard deviation. Recall that for a dataset consisting of

observations on one variable, , the -scores are , where and are the arithmetic mean and standard deviation of the observations. The -scores have been used for instance to detect outliers, though they have a number of important limitations (Barnett & Lewis, 1994). The mean of is zero while both its standard deviation and variance are equal to 1. Also, is both the standardized distance between and and the Euclidean distance between and , whereas the Euclidean distance between and , , is equal to , the standardized distance between and .

Consider now a multivariate dataset and imagine a standardizing transformation with similar properties as the -scores transformation. Computing the -scores separately for each variable does not achieve that desideratum unless all the variables are uncorrelated (that is, in case both and are diagonal matrices). It turns out that the multivariate transformation which standardizes a multivariate dataset taking into account the correlations between all the variables is the Mahalanobis transformation, defined in matrix notation by , where

is a vector containing the

measurements of the th object, is the mean vector, containing the means of the individual variables and is a square root of the inverse of . This transformation has the same form of the -scores transformation, but uses matrices in place of single values. It is easy to verify that the standardized data matrix, , formed with the vectors, is such that all the variables have zero mean and unit variance, and all pairwise covariances are zero (in other words, the mean vector of

is the null vector, and both its covariance and correlation matrices are the identity matrix,

). The Euclidean distance between a standardized observation, , and the null vector is , which can be written in matrix notation as and defines the Mahalanobis distance between the observation and the mean of the observations, . The Euclidean distance between two standardized observations, and , is and defines the Mahalanobis distance between and , . Like the -scores, Mahalanobis distances to the mean have been used to detect outliers in multivariate datasets (Gnanadesikan & Kettenring, 1972). Mahalanobis distances between two objects are useful, e.g., in clustering applications.

2.2 When p is large and n is small

Before we discuss the very complicated issue of the visualization of this kind of data, let us consider an apparently naive question: can we use Mahalanobis distance when the number of variables is larger than the number of observations? A first quick answer would be: no, because when the covariance matrix can not be inverted ( is singular) and therefore Mahalanobis distance is not defined. However, it is still possible to define the Mahalanobis distance by reasoning as follows. Three points (in general position) in a 3-d space (, ) define a plane, in other words, they lie on a certain 2-d subspace. If we adopt a coordinate system in that subspace, Mahalanobis distances can be computed, because the new covariance matrix will not be singular. A similar argument can be applied to higher dimensional spaces. A dataset with observations in variables, with , can be represented, without any loss of information, in a new set of variables, for which Mahalanobis distances can be computed. Moreover, such a set of variables is easy to find. Consider, for instance, the principal components of corresponding to non-null eigenvalues of . These new variables are linear combinations of the original variables and it is easy to move from one system to the other. The Mahalanobis distance defined in this way coincides with the generalized Mahalanobis distance (Mardia, 1977), for which the non-existing inverse is replaced by the Moore–Penrose generalized inverse. Therefore we conclude that we can still standardize datasets with , by applying the Mahalanobis transformation in the largest subspace for which the covariance matrix is invertible, that is, using new variables which are linear combinations of the original variables. But when we perform the computations just described we arrive at the following surprising result (details and proofs are given in Appendix A).

Theorem 1

For every dataset with points, , and variables we have that: (i) , for every , and (ii)  for every . In other words, whatever the points , as long as , the standardized data always form a regular pattern in which the distance from every point to the center is a constant and the distance between any two points is another constant, both constants depending only on .

Despite Mahalanobis distance has been known and used for more than 70 years, these apparently simple results were not found in the literature. Incidentally, we are also able to show, following almost the same proof as in Theorem 1, that for every dataset, irrespective of the number of observations or variables, we have that (iii) , for every , and (iv)  for every (a proof of (iii) was published in Trenkler & Puntanen (2005) and also in Gath & Hayes (2006). The results given in Theorem 1 can also be connected to asymptotic results in Ahn et al. (2007).

Figure 1 illustrates Theorem 1 for datasets with 3 points and a number of variables larger or equal than 2. This is the only case we can represent directly in a 2-d plot (the plane containing the three points), and we can see that every dataset with 3 points in variables corresponds, when standardized and apart from an arbitrary rotation, to the “same” equilateral triangle (the general position assumption excludes cases where the 3 points lie on a straight line). Similarly, for points in variables, every dataset corresponds, when standardized, to the “same” regular tetrahedron (again apart from an arbitrary rotation). For we have to imagine a regular geometric object in a dimensional space which is the appropriate member in the sequence: equilateral triangle, regular tetrahedron, …These objects are called regular simplices (triangle2-simplex; tetrahedron3-simplex; pentachoron4-simplex; …; -simplex; …). By the above result, we can envisage the standardized version of an data matrix, with , as the vertices of a regular -simplex, such that the length of every edge is , and every vertex is located at a distance of

from the center of the simplex. This is a very simple regular structure with all the points at the boundary of its convex hull (the simplex) while the interior of the convex hull is completely empty. This last property is also shared by the original data, because they can be seen as a non-singular linear transformation of the standardized data:

.

Figure 1: Three datasets with 3 random points in dimension 2. (A) Original observations, , and . (B) Standardized observations, , .

We have thus shown that the point cloud of a multivariate dataset with is like an empty shell which, when standardized, looks the same whatever the data. This reality has strong implications. If we insist on considering the multivariate relation between all the variables, it is not possible, unless external information is provided, to: (i) separate outliers from non outlier observations, (ii) detect any kind of deviations from symmetric structures, (iii) distinguish between linear and non linear structures, or (iv) identify any type of clustering.

Let us now consider projections of these data structures, that is, imagine traveling “around” the point cloud and taking a large number of interesting pictures. The next result shows that these pictures can show virtually anything we want to see and have therefore to be used very carefully when extracting conclusions about the data.

Theorem 2

For every matrix with points and variables, , and every non-singular two dimensional arrangement of points () it is always possible to find (explicitly) an orthogonal projection of which is “similar” to .

The result given in this theorem can be connected to some results in the mathematics literature (Baryshnikov & Vitale, 1994; Eastwood & Penrose, 2000). Remarkably, the “piling effect”, characterized by the existence of directions such that projections of data onto those directions take only two distinct values (Ahn & Marron, 2010), is just a special case of Theorem 2.

Figure 2 illustrates Theorem 2 using two well known datasets: the colon cancer microarray data (Alon et al., 1999) and the ORL face database (Samaria & Harter, 1994). The implications for data analysis are the same as for Theorem 1: if there is no external information, there is little we can conclude about the multivariate structure of an high dimensional dataset. For instance, is a projection with an outlier evidence of the existence of an outlier? It can not be, because we can see such a projection in every high dimensional dataset with just the same number of points, irrespective of the presence of any outlying observation.

Figure 2: Illustration of Theorem 2. (A) and (C) Arbitrary configurations of points in 2 dimensions. (B) An orthogonal projection of high dimensional data which is similar to the configuration in (A), where the data comes from the ORL face database (Samaria & Harter, 1994) whith observations (pictures) on variables (pixels). (D) An orthogonal projection of high dimensional data which is similar to the configuration in (C), where the data comes from the colon cancer microarray dataset (Alon et al., 1999) whith observations (arrays) on variables (genes); the black circles correspond to the 22 normal tissue samples and the white circles correspond to the 40 tumor tissue samples (the arbitrary configuration in (C) specifies separation of the two groups).

We now know that the

data space is quite odd. It is then natural to ask whether those singularities appear suddenly when

reaches or whether the properties of the space change gradually as approaches from bellow. The upper bound of the Mahalanobis distance, together with the fact that the expected value of is for , whatever the distribution of the data (Mardia, 1977), show that a transition must start far before reaches (this may also be related to Corollary 1.1 in Donoho & Tanner, 2005

, which states a rigorous result about such transitions for samples from a multivariate normal distribution)

. Figure 3 illustrates this aspect. The plots show orthogonal projections of the colon cancer microarray dataset, selected to be as similar as possible to configuration (C) of Figure 2, but using only a subset with randomly chosen variables. Contrary to what happened in Figure 2, it is no longer possible to replicate any configuration, however, certain aspects of the given structure remain visible, even in the case , where the separation of the two groups is still noticeable. This example shows that it may be dangerous to interpret motifs seen in projections if the ratio is not large enough.

Figure 3: Six orthogonal projections of the colon cancer microarray dataset, with p n, selected to be as similar as possible to the configuration (C) of Figure 2. From top to bottom, left to right: p = 60, 50, 40, 30, 20 and 10 randomly selected variables.

Taking into account the results presented one should investigate many recent scientific discoveries in various areas which relied on the analysis of high dimensional datasets. This can be easily done by (i) generating a number of artificial datasets similar to the dataset under study, for instance, with the same number of variables and observations, or with equal means and variances, (ii) studying those artificial datasets in the same manner as the original data, (iii) comparing the results. One example of the application of this strategy is presented next, but others could be given.

2.3 Example

In this example we consider again the colon cancer microarray dataset and assess the performance of several outlier detecting methods recently proposed and advertised as appropriate for high dimensional datasets

(there is an extensive literature about the outliers of this dataset, see Shieh & Hung, 2009, and the references therein). The methods considered are the ROBPCA of Hubert et al. (2005), three methods proposed in Filzmoser, Maronna & Werner (2008), denominated by the authors as PCOUT, SIGN1 and SIGN2, and the method proposed in Shieh & Hung (2009), referred to as SH. More details about this example are provided as Supplementary Material.

We first concluded that all the methods point out some observations as outliers (ROBPCA: 5 normal and 10 tumor; PCOUT: 1 normal and 6 tumor; SIGN1: 7 normal and 21 tumor; SIGN2: 4 normal and 8 tumor; SH: 3 normal and 5 tumor), unfortunately they are unanimous only about two cases (one normal and one tumor).

In order to show that the flagged outliers may well be illusions created by the high dimensionality, we simulated 500 datasets with multivariate normal distribution but similar to the original colon cancer dataset (that is, with two groups, same number of variables and observations, same means and covariances per group). We then applied the five outlier detecting methods to each dataset and registered the number of outliers detected. The median number of detections observed were the following: in the group of 22 observations, ROBPCA: 6, PCOUT: 3, SIGN1: 8, SIGN2: 1, SH: 2, and in the group of 40 observations, ROBPCA: 12, PCOUT: 4, SIGN1: 17, SIGN2: 2, SH: 2. These figures are far too large than would be anticipated and are similar to the number of outliers “detected” in the real data. We must analyze the results deeper.

All the methods are calibrated at the normal, meaning that the probability of declaring an observation from a normal distribution as an outlier, is pre-specified at a certain level,

. In this case for all the methods and both the real and simulated data. According to this definition, the number of outliers detected in a sample of observations from a normal distribution follows a binomial distribution. Therefore, we expect that the sample containing the number of outliers found in each of the simulated datasets to behave as a sample with 500 observations from a binomial(40,0.025) or a binomial(22,0.025). From now on we refer only to the part of the simulation with , as the results and conclusions from the other part () were similar.

The plots in the left column of Figure 4 show the observed frequency distributions of the number of outliers detected with and all the variables () included. The ’+’ symbol in the plots indicate the expected frequencies under the binomial(40,0.025) distribution. All the observed frequencies are very different from the expected frequencies. We may conclude that the detection methods are not working as they should, especially the first three, ROBPCA, PCOUT and SIGN1, and are probably providing misleading results also for the real data.

We are convinced that the responsibility for the failure of the outlier detection methods can be ascribed, at least partially, to the high dimensionality of the data. To support this claim we repeated the simulation with normal data for a much smaller number of variables. The illustration described in Figure 3 shows that may already be too large when . Then we have selected randomly variables from the original 2000. The plots with the number of outliers detected are shown on the right column in Figure 4.

Figure 4: Results of the detection of outliers by various methods (ROBPCA, PCOUT, SIGN1, SIGN2 and SH, see text for details) in 500 simulated multivariate normal data sets with n = 40 observations and p = 2000 variables (left) and p = 5 variables (right). The ’+’ signs represent expected frequencies under a binomial(40,0.025) distribution.

We conclude that, except for the PCOUT method, all the observed frequencies have moved closer to the expected frequencies and in the case of ROBPCA and SIGN1 there is in fact a fantastic recovery. In this low dimensional situation all the five methods but one are in agreement and doing more or less close to what they are supposed to do.

As we wondered what could possibly justify the observed behaviour of the five methods we looked into the algorithmic details of each method and can then add the following explanation for the three types of behaviour observed.

Methods that work fine in low dimensions and fail in high dimensions (ROBPCA and SIGN1): when both methods work internally with the data rotated to its proper subspace (of dimension ) and then look for projections showing outliers. But, an easy application of Theorem 2 to one-dimensional projections shows that (as long as

) there is at least one projection where each and every observation sticks out from the remaining, which are projected all to the same point. This makes the usual criterion based on robust z-scores (standardized observations using the median and the median absolute deviation in place of the mean and the standard deviation) or the more sophisticated Stahel–Donoho outlyingness measure (for each observation this is the maximum of all possible robust z-scores of that observation, taken along all possible projections) completely useless. When the number of variables is small (

) there are no longer such “arbitrary” projections and both methods work.

Methods that work fine in low and high dimensions (SIGN2 and SH): SIGN2 is similar to SIGN1, and SH is similar to ROBPCA. However, in both cases a dimension reduction by principal components is the first step of the procedures. This is then followed by outlier detection in the reduced space (the number of principal components to retain is chosen by fixing a proportion of explained variance, in the case of SIGN1, or by an automatic selection method based on the scree plot, in the case of SH). By bringing the high dimensional situation to the usual the difficulties described above are avoided, and therefore the methods work as expected no matter the dimensionality.

Method with problems both in high and low dimensions (PCOUT): according to Filzmoser, Maronna & Werner (2008), PCOUT was designed to work in high dimension cases. This explains why it does not work very well in low dimensions. However, in the high dimensional situation, the method uses robust versions of the Mahalanobis distance, which, in the light of Theorem 1 is very dangerous, and may explain the problems encountered.

This example clearly shows that knowledge of the geometric properties of high dimensional data, in particular the results given in this paper, is of crucial importance for the correct development of methods to deal with this type of data.

3 Conclusions

New technologies are great in providing floods of data full of information and potential knowledge, but the extraction of such information can often prove very difficult. High dimensional spaces are typical in revealing such difficulties that mathematicians label curse of dimensionality. In the present work we focus on spaces where the number of variables () is at least the number of observations (), a case that occurs, in statistical practice, mostly within the high dimensional data framework. We look at the behaviour of the Mahalanobis distance and the idea of projection, so essentials in the analysis of multivariate data.

When we prove that, the Mahalanobis distance becomes degenerated and its known distance properties are lost. Under the same restriction it is confirmed that the idea of projection loses interest and becomes mostly useless. These conclusions imply that many procedures for analyzing data will not work in these spaces.

Although we have seen users insisting in using artifacts to analyze this kind of data, it seems that they are not aware of the geometric complexities of spaces under these conditions. Disclosing what happens to the Mahalanobis distance and to the usual system of projections we gain insight into the geometric properties of those spaces and hopefully contribute to: (i) prevent unconscious applications of inadequate methods and (ii) help to devise new methods where those geometric properties have to be taken into account.

Having said this, it should be understood that we are not suggesting that traditional statistical methods are not worthwhile and their use should be abandoned when analyzing high dimensional data. On the contrary, we consider that this formidable source of statistical knowledge should not be forgotten but used properly. To allow traditional methods to operate one first step towards reducing the dimensionality of the data should be given. For dimensionality reduction to be achieved one can think of a number of techniques including a panoply of variable selection and regularization procedures.

Acknowledgement

This work received financial support from Portuguese National Funds through Fundação para a Ciência e a Tecnologia.

Appendix

Definitions and notation

Definition 1 (notation)

Data matrix (): .

Definition 2 (points in general position)

The p-dimensional observations in the () data matrix are said to be in general position if has maximal rank, that is, if . This property holds with probability one for continuous variables.

Definition 3 (further notation)

Mean of (): .

Centered data matrix (): ( denotes a vector with all its elements equal to ).

Covariance matrix of (): . If is in general position then rankrank.

Standardized data matrix (): , where is any square root of (which means that ).

Mahalanobis distance between an observation, , and : .

Mahalanobis distance between two observations, and : .

Matrix ():

(1)

where . Note that and .

Augmented data matrix (): , which verifies

(2)

where is the well known “hat matrix” (Hoaglin & Welsch, 1978).

Main results

Theorem 1

For every dataset with points, (in general position), and variables we have that: (i) , for every , and (ii)  for every .

Proof: For , is singular and must be replaced by a generalized inverse (). This is equivalent to reducing the number of variables to (e.g., the standardized non-trivial principal components). Therefore we only have to prove the case.

In that case, has dimension and rank , whereas has dimension and rank , so it can be inverted, resulting in

which is equivalent to

(3)

That is, , for all , which proves (i). On the other hand, from (3), for any , , therefore

which proves (ii).

Theorem 1 (extra results)

For every dataset with points, , and variables we have that: (iii) , for every , and (iv)  for every .

Proof: (iii) Equation (2) writes , and applying the well known result (Hoaglin & Welsch, 1978), , the conclusion that follows immediately (this is a short alternative proof of Theorem 2.1 in Gath & Hayes, 2006).

(iv) Define and check that is symmetric and idempotent. Consider the th and th row (or column) vectors of , and , with . Then and (we need the assumption that , for all , which is not restrictive, because the cases where are considered in (ii)). Let be the angle between and , then

implying that . Considering this result and the inequality , it follows that

which is equivalent to . Finally, simple manipulations of the definitions yield

Theorem 2

For every dataset with points and variables, , and every non-singular two dimensional arrangement of points () it is always possible to find (explicitly) an orthogonal projection of which is “similar” to . More precisely, given and , there is an orthogonal transformation, characterized by the matrix

, whose columns are orthonormal vectors, such that

, where is an affine transformation of (that is, , for some non-singular matrix and some two dimensional vector ).

Proof: Without loss of generality we assume that is such that and . As in the proof of Theorem 1 we need to consider only the case , for which the covariance matrix of , , is invertible. Let , be the standardized data matrix, and consider the matrix defined by . We show next that (i) the columns of are orthonormal vectors, and (ii) .

(i) , which follows from equation (3) and by the assumption that is standardized.

(ii) , for the same reasons.

The next step is to rewrite (ii) in terms of or, equivalently, in terms of , . Replacing, in the last equality, the matrix

by its singular value decomposition,

, where the columns of () and () are orthonormal and is a diagonal matrix, leads to , which is equivalent to . Therefore, and concluding the proof, , and .

Remark 1

The two-dimensional projections are usually the most interesting to consider, but the theorem could be generalized easily to projections on a subspace of arbitrary dimension, . The case would contain the “piling effect” as a special case, and also the curious case of projections where all but one point (which can be any of the points) coincide.

Remark 2

The key to prove both theorems is the fact that for .

Remark 3

The plots in Figure 2, produced with the formulas given in this theorem, illustrate the kind of “similarity” between and that can be obtained when , no matter the data, and that can be described as perfect “similarity”. On the contrary, in the cases considered in Figure 3 where , perfect “similarity” is no longer possible. To produce plots as “similar” as possible to we used a least squares criterion and look for the transformation, , minimizing , concluding that must be replaced by (all the other formulas are unchanged).

References

  • Ahn & Marron (2010) Ahn, J. & Marron, J. S. (2010). Maximal data piling in discrimination. Biometrika 97, 254–259.
  • Ahn et al. (2007) Ahn, J., Marron, J. S., Muller, K. M. & Chi, Y.-Y. (2007). The high-dimension, low-sample-size geometric representation holds under mild conditions. Biometrika 94, 760–766.
  • Alon et al. (1999) Alon, U., Barkai, N., Notterman, D. A., Gish, K., Ybarra, S., Mack, D. & Levine, A. J. (1999).

    Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays.

    Proc. Natl. Acad. Sci. 96, 6745–6750.
  • Barnett & Lewis (1994) Barnett, V. & Lewis, T. (1994). Outliers in Statistical Data. 3rd edn. Chichester: Wiley.
  • Baryshnikov & Vitale (1994) Baryshnikov, Y. N. & Vitale, R. A. (1994). Regular simplices and Gaussian samples. Discrete Comput. Geom. 11, 141–147.
  • Bellman (1957) Bellman, R. E. (1957). Dynamic Programming. Princeton: Princeton University Press.
  • Clarke et al. (2008) Clarke, R., Ressom, H. W., Wang, A., Xuan, J., Liu, M. C., Gehan, E. A. & Wang, Y. (2008). The properties of high-dimensional data spaces: implications for exploring gene and protein expression data. Nature Rev. Cancer 8, 37–49.
  • Donoho & Tanner (2005) Donoho, D. L. & Tanner, J. W. (2005). Neighborliness of randomly-projected simplices in high dimensions. Proc. Natl. Acad. Sci. 102, 9452–9457.
  • Eastwood & Penrose (2000) Eastwood, M. & Penrose, R. (2000). Drawing with complex numbers. The Mathematical Intelligencer 22, 8–13.
  • Filzmoser, Maronna & Werner (2008) Filzmoser, P., Maronna, R. & Werner, M. (2008). Outlier identification in high dimensions. Computational Statistics and Data Analysis 52, 1694–1711.
  • Gath & Hayes (2006) Gath, E. G. & Hayes, K. (2006). Bounds for the largest Mahalanobis distance. Linear Algebra and its Applications 419, 93–106.
  • Gnanadesikan & Kettenring (1972) Gnanadesikan, R. & Kettenring, J. R. (1972).

    Robust estimates, residuals, and outlier detection with multiresponse data.

    Biometrics 28, 81–124.
  • Hastie, Tibshirani & Friedman (2009) Hastie, T., Tibshirani, R. & Friedman, J. H. (2009). The Elements of Statistical Learning. 2nd edn. New York: Springer.
  • Hoaglin & Welsch (1978) Hoaglin, D. C. & Welsch, R. E. (1978). The hat matrix in regression and ANOVA. The American Statistician 32, 17–22.
  • Hotelling (1933) Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24, 417–441.
  • Hubert et al. (2005) Hubert, M., Rousseeuw, P. J. & Vanden Branden, K. (2005).

    ROBPCA: a new approach to robust principal component analysis.

    Technometrics 47, 64–79.
  • Johnson & Wichern (2007) Johnson, R. A. & Wichern, D. W. (2007). Applied Multivariate Statistical Analysis. 6th edn. New Jersey: Prentice Hall.
  • Johnstone & Titterington (2009) Johnstone, I. M. & Titterington, D. M. (2009). Statistical challenges of high-dimensional data. Phil. Trans. R. Soc. A 367, 4237–4253.
  • Mahalanobis (1936) Mahalanobis, P. C. (1936). On the generalised distance in statistics. Proceedings of the National Institute of Sciences of India 2, 49–55.
  • Mardia (1977) Mardia, K. V. (1977). Mahalanobis distances and angles. Multivariate Analysis IV (ed P. R. Krishnaiah), pp. 495–511. Amsterdam: North-Holland
  • R Development Core Team (2012) R Development Core Team (2012). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0, http://www.R-project.org/.
  • Samaria & Harter (1994) Samaria, F. & Harter, H. (1994). Parameterisation of a stochastic model for human face identification. Proc. of the 2nd IEEE Workshop on Appl. of Comp. Vision, 138–142.
  • Shieh & Hung (2009) Shieh, A. D. & Hung, Y. S. (2009). Detecting outlier samples in microarray data. Statistical Applications in Genetics and Molecular Biology 8, 13.
  • Trenkler & Puntanen (2005) Trenkler, G. & Puntanen, S. (2005). A multivariate version of Samuelson’s inequality. Linear Algebra and its Applications 410, 143–149.