The space of projective shapes of landmarks in -dimensional real projective space
is of interest in computer vision. It is commonly defined as the topological quotient ofcopies of modulo the landmark-wise action of the projective linear group . This space arises naturally in the single view uncalibrated pinhole camera model: when taking a -dimensional picture in of a
-dimensional object without knowledge of any camera parameters such as focal length, angle between the object hyperplane and film hyperplane, etc., then the original object can only be reconstructed up to a projective transformation. Similarly, it arises in the multiple view uncalibrated pinhole camera model: when taking multiple-dimensional pictures of an object in the original configuration of landmarks can only be reconstructed up to a projective transformation. For details, we refer the reader to the literature, e.g. [3, 5].
Other space of interest in computer vision include similarity and affine shape spaces. In shape spaces, one would often like to make metric comparisons, which requires e.g. the structure of a Riemannian manifold. For affine or similarity shapes, the topology of the shape space is well understood and there are natural choices for a Riemannian metric. Similarity shape space is a CW complex after removing the trivial shape , while affine shape space has a naturally ordered stratification with each stratum being diffeomorphic to a Grassmannian [4, 10]. In both cases, the topological subspace of shapes with trivial isotropy group, i.e. the shape space of the configurations on which the group action is free, has a natural structure of a Riemannian manifold.
In the case of projective shapes, it turns out that the topological subspace of shapes with trivial isotropy group cannot be given the structure of a Riemannian manifold since it is only a differentiable T1 manifold, but not Hausdorff. Hence, we have to look for other topological subspaces, which can be endowed with a Riemannian metric. This search is the main purpose of this article.
Besides the quest for a Riemannian structure, there are more desirable properties for a “good” topological subspace:
it should be a manifold with complete Riemannian metric;
it should be closed and the Riemannian metric invariant under reordering of the landmarks in the configuration (relabeling);
when containing a degenerate shape, i.e. a shape with non-trivial projective subspace constraints (see Section 2), it should also contain all less degenerate shapes; we will then say that the topological subspace respects the hierarchy of projective subspace constraints;
it should contain as many shapes as possible in the sense that adding further shapes results in the violation of at least one of the properties (maximality).
To our knowledge, there are only two established ways to obtain topological subspaces fulfilling some of these properties, which will be discussed in Section 3. Firstly, one can take only those shapes whose first landmarks are in general position and thus form a so-called projective frame. This topological subspace is homeomorphic to copies of ; in particular it respects the hierarchy of projective subspace constraints while being maximal, Hausdorff and a differentiable manifold, i.e. locally Euclidean with smooth transition maps and second-countable. Unfortunately, it is not closed under relabeling. Secondly, one can take all those shapes whose projective subspace constraints fulfill a certain regularity condition, called Tyler (fully-)regular . This topological subspace is Hausdorff, closed under relabeling, respects the hierarchy of projective subspace constraints and, as we show in Section 5, a differentiable manifold. Recall that a Riemannian metric can be defined on any differentiable Hausdorff manifold . However, these topological subspaces have been constructed in an ad hoc fashion. As of now there is no systematic approach to obtain “good” topological subspaces based on the geometrical and topological properties of projective shape space.
In this paper, we therefore analyze the topology of projective shape space in detail. After recalling some basic facts, fixing our notation in Section 2, and discussing prior approaches in Section 3, we show which shapes can be separated from each other in the T1 sense, i.e., either one is not contained in some open neighborhood of the other, in Section 4. In particular, we will show that the subspace of shapes with trivial isotropy group is T1 and a differentiable manifold. Since there are free shapes without a frame, they do not suffice to construct charts. We thus generalize the notion of a frame to obtain charts. In Section 5, we show that two shapes which cannot be separated in the Hausdorff sense are already degenerate in a particular way. This allows us to characterize a reasonable family of differentiable Hausdorff manifolds in Section 6 which additionally possess properties (b), (c), and (d). In Section 7, we give a geometric justification for Tyler standardization of Tyler regular shapes introduced by Kent and Mardia  and a Riemannian metric on this topological subspace.
2 Preliminaries and notation
For real projective space is defined as the topological quotient of modulo the multiplicative group so it can be seen as the space of lines through the origin in . A projective subspace of of dimension is then the set of lines lying in an -dimensional linear subspace of . Analogously, one can define the projective span of points in as the set of lines lying in the linear span of some representatives of the points in .
There is a natural, well-defined action of the general linear group on by letting it act on representatives in . Since the action of a matrix on does not change when multiplying the matrix by a non-zero scalar, the action of is identical with the action of the projective linear group . This action is naturally carried forward to the product space of configurations
by letting it act component-wise. Note that projective transformations, i.e. the elements of , map projective subspaces of to projective subspaces of the same dimension, i.e. points to points, lines to lines etc. So, if is a configuration with three landmarks on a line, then the images of these three landmarks under a projective transformation also lie on a line.
For and , the space of projective shapes of landmarks in is defined to be the quotient space
together with the quotient topology. Since the projection map is open, the topology of is also second countable; it thus can be characterized by sequences, just like . Further, we can represent a configuration in homogeneous coordinates: up to left-multiplication with a diagonal -matrix with non-zero real entries, the landmarks in can be represented as a real -matrix whose non-trivial rows , represent the landmarks in . The corresponding equivalence class , i.e. the shape, consists of all matrices of the form with being a non-singular diagonal -matrix, a non-singular -matrix, i.e.,
Throughout this article, we denote a configuration by a lower case letter, its matrix representation by the corresponding upper case letter and the shape of resp. by resp. . In abuse of language, we will call a configuration, too. Further, we define the rank of a configuration to be the rank of any corresponding matrix . Note that the rank is invariant under .
Our aim is to find topological subspaces of that can be given the structure of a Riemannian manifold. Topologically speaking, these topological subspaces need to be differentiable Hausdorff manifolds, as those can be given the structure of a Riemannian manifold .
Unfortunately, the space of all projective shapes is not a differentiable Hausdorff manifold, and indeed it is not even T1. This is easily seen by considering the open neighborhoods of the trivial shape where all landmarks coincide. Any open neighborhood of the trivial shape is actually already the full space . This phenomenon occurs in similarity and affine shape space as well.
Before we turn to analyze in detail, we define some topological subspaces of (resp. ):
which contains a configuration if and only if the landmarks are in general position, i.e., no -dimensional projective subspace of with contains more than of the landmarks, i.e., any of the landmarks in span . An element of is called a (projective) frame. Note that is dense in .
which contains a configuration if and only if it contains at least one frame, i.e., if and only if there exists a permutation of the landmarks such that thus [9, Remark 2.1; being called there].
which contains a configuration if and only if it has trivial isotropy group, i.e., Elements with trivial isotropy group are called free or regular. Note that as shown by Mardia and Patrangenaru .
which contains a configuration if and only if it is splittable, i.e., there is a subset s.t. where and denotes the restriction of to landmarks with index
which contains a configuration if and only if it is of full rank, i.e., there is no projective subspace of dimension which contains all landmarks. Note that (take ).
which contains a configuration if and only if any -dimensional projective subspace of , contains fewer than landmarks. These configurations are called Tyler (fully-)regular by Kent and Mardia .
Note that we always denote the set of equivalence classes by a lower case letter, the corresponding set of configurations by an upper case letter, for example etc. for the configuration spaces, etc. for the corresponding shape spaces.
We say that a configuration fulfills the projective subspace constraint for a subset of size if and only if there is a projective subspace of dimension such that for all i.e. We denote the collection of projective subspace constraints fulfilled by a configuration by . We call a projective subspace constraint trivial if is a subset of size , and non-trivial otherwise. Further, we call splittable in if there are with , . Thus a configuration is splittable, i.e., , if and only if is splittable (slightly generalizing our notation). We noted before that is invariant under , i.e., for all , whence is a property of the projective shape
3 Previous approaches
The first statistical approach to projective shape space is via frames, which are a well-known concept in projective geometry. As mentioned before, a frame is an ordered set of landmarks in general position. The group action of is both transitive and free on the space of frames, i.e., for any two frames there is a unique projective transformation mapping one frame to the other . This quickly leads to the following result by mapping the frame in the first landmarks to a fixed frame:
is homeomorphic to
So, is a differentiable Hausdorff manifold and respects the hierarchy of projective subspace constraints, but is not closed under relabeling. The closure of under permutations is—by definition—the topological subspace of shapes with a frame. is a differentiable manifold, but not Hausdorff for any , as we will see in Proposition 5.
is homeomorphic to a -dimensional differentiable T1 manifold.
Lemma 3 gives homeomorphisms from the topological subspaces of shapes with a frame in a fixed subset of landmarks to . Further, note that these topological subspaces of shapes with a frame in a fixed subset of landmarks are open in and . Hence, these homeomorphism are “manifold-valued” charts on . Ordinary charts on can easily be obtained by composition with charts on the manifold , e.g. inhomogeneous coordinates. These charts are compatible since the transition maps are just multiplications with non-singular matrices as well as division by non-vanishing parameters depending smoothly on the representation matrix.
Alternatively, one can consider the space of shapes in general position which is also a differentiable Hausdorff manifold, respects the hierarchy of projective subspace constraints, and is closed under relabeling. The drawback of is that it is not maximal for any ,
For , a frame consists of three distinct landmarks. Hence, consists of all configurations with distinct first three landmarks and arbitrary fourth landmark. is then homeomorphic to the real projective line or—equivalently—the circle. Meanwhile, consists of all configurations with at least three of its landmarks distinct and thus forming a frame. is homeomorphic to a circle with three double points corresponding to the single pair coincidences, which cannot be separated in the Hausdorff sense . Finally, consists of configurations with no landmark coincidences, hence is homeomorphic to the circle with three points removed.
A different approach was developed by Kent and Mardia . The space of Tyler regular configurations comprises configurations all of whose projective subspace constraints satisfy the inequality . It was shown that any Tyler regular configuration has a matrix representation fulfilling
with denoting the
-dimensional identity matrix. This so-called Tyler standardizationis unique up to multiplication of the rows by
and right-multiplication by an orthogonal matrix, i.e. unique up to a compact group action, and can be viewed as a projective pre-shape. By considering, one can even remove the ambiguity of the -action. This gives a covering space of the space of Tyler regular shapes. The covering space is Hausdorff, whence is Hausdorff.
We show in Section 7, that is a differentiable Hausdorff manifold; it is obviously closed under relabeling and respects the hierarchy of projective subspace constraints. Additionally, we show that is maximal for some, but not all and . Note that the approach via frames differs from the approach via Tyler regularity since, for , there are Tyler regular shapes without a frame, see Figure 1.
In the case and , consists of shapes with projective subspace constraints with and , i.e. the shapes in general position. Hence, with being homeomorphic to the circle with three points removed, as we have seen before.
Neither of these approaches discusses the topological background of these choices. The goal of this article is to shed light on the topology of these topological subspaces of projective shapes.
4 The manifold of the free
To understand a topology of a topological space , it is vital to know which elements of cannot be separated from another by open neighborhoods. It is common to use the well-known separation axioms to described the degree of separation. Two of those will be discussed here.
A topological space is said to be
if for any two points there are open neighborhoods and of and respectively not containing the other point, i.e., and .
- Hausdorff or
if for any two points there are disjoint open neighborhoods of and .
The intersection of all open neighborhoods to a point is a useful tool towards understanding the separation properties of a space . This set was introduced as the blur of in by Groisser and Tagare in their discussion of affine shape space . We will call a point unblurry if and blurry in the case that its blur is a strict superset of
Equivalently, the blur could also be defined via sequences.
Let be a topological space and Then, if and only if the constant sequence converges to
if and only if is in every neighborhood of which happens if and only if the sequence converges to
This concept is closely related to the more familiar concept of closure which has also been pointed out by Groisser and Tagare .
[4, Lemma 5.2] Let be a topological space and Then, if and only if the latter denoting the closure of in
In particular, every point is unblurry if and only if every point is closed, which in turn is equivalent to the space being T1 . This motivates us to take a closer look at the unblurry shapes.
As it turns out, a shape is blurry if it is splittable; the converse is also true as we will show after Theorem 4.
Let be a splittable shape. Then is blurry.
We will use Lemma 4.
First, consider an arbitrary shape with .
There is a non-singular matrix such that for some and being a column vector of
being a column vector ofzeroes. Of course, is still of shape . Then, the sequence with and arbitrary has limit . Hence, for any while there is a such that Therefore, , whence is blurry
Now, let be of rank with . W.l.o.g. else . Then there is a suitable permutation of the rows of and a suitable non-singular matrix such that the matrix is a block diagonal matrix
for some matrices and . The sequence given by
has limit for any Hence,
Again, there is a which breaks a projective subspace constraint of , whence and consequently and is blurry.
In the case and , the topological subspace of splittable configurations consists of the trivial configurations (all landmarks identical) and all those comprising of only two different landmarks, when either three landmarks coincide or there are two pairs of landmarks coinciding. The blur of the shape with and (double pair coincidence) comprises of and the single pair coincidences, i.e. shapes, with and with
Due to Proposition 4, we henceforth limit ourselves to the analysis of those configurations (resp. shapes) which are not splittable. Those can be characterized algebraically via the group action.
A configuration is free if and only if it is not splittable, i.e.
If then is obviously splittable, but not free. Hence, we will focus on configurations with
Now, assume there are projective subspace constraints such that . Then there is a permutation of the rows and a matrix such that is a block diagonal matrix . Hence, is not free since
Therefore, is not free, henceforth neither is .
For the opposite direction, assume is not free. Then there exists a diagonal matrix and some such that .
Hence, the rows of are eigenvectors of with corresponding eigenvalues
are eigenvectors of
with corresponding eigenvalues, say (taking at most distinct eigenvalues). There are at least two distinct eigenvalues, else contradicting the assumption. Then, with , while whence is splittable.
In the case , the splittable shapes are those comprising of at most two distinct landmarks as we have seen before. Thus, Proposition 4 states that a shape is free if and only if it has at least three distinct landmarks. Three distinct landmarks always form a frame for . Indeed, Mardia and Patrangenaru  have shown for any that shapes which include a frame are free, i.e. . However, the other inclusion does not hold for : e.g. for , take three lines, which are not coplanar, but have a common intersection point, and put two landmarks on each line, and another on the intersection point. Such a configuration of 7 landmarks is free, but does not contain a frame since there are no 5 landmarks in general position, see Figure 1(a). The same argument works when removing the landmark on intersection point. Analogously, a free shape without a frame can be constructed for any
Hence, having a frame is not essential for a shape to be free. While frames can be used as charts on , this is not possible for for since the charts associated with frames do not cover for . However, the notion of a frame can be generalized to obtain charts on as follows.
A free configuration contains at least landmarks in general position since a free configuration is of full rank. Now, a configuration whose first landmarks , say, are in general position, i.e. , is equivalent to a matrix of the form
where consists of non-trivial rows. For such a configuration define its corresponding (undirected) graph by taking the columns of as vertices, i.e. and there exists an edge labeled with “” between the vertices if both and (see Figure 1(b) as an example). Note that there may be multiple edges of different colors between two vertices.
This definition of the graph of a configuration with the first landmarks in general position is well-defined and invariant under : let be an equivalent configuration, be the upper left square block of with rows, be the lower right square block of with rows, be the first landmarks of , be the last landmarks of . Then in is given by
Hence, is unique up to left- and right-multiplication by non-singular diagonal matrices. But these actions do not affect the graph.
In fact, can be seen as an edge-colored graph. The set of edges has a partition into sets of edges labeled with “color”
This definition can easily be extended to any configuration with a given set of landmarks in general position.
Now, we can connect freeness with graph properties.
Let be a configuration whose first landmarks are in general position. Then is free if and only if is connected.
If is not connected, then the columns of split into two disconnected sets, so is splittable, as is , hence not free according to Proposition 4.
for the first rows of For any two connected columns there is a row such that both and Hence,
where is the -th row vector of the standard basis of . From this we conclude
and thus since all columns are connected, so and , i.e., is free.
In the following, we will call landmarks in general position together with a connected tree with edges labeled with the remaining landmarks a pseudo-frame. So contains no cycles and gets disconnected if an edge is removed whence it is a minimal substructure of a connected graph. This generalizes the idea of a “frame” since a frame is a pseudo-frame with a connected tree on landmarks in general position where all edges are labeled with the same landmark (see Figure 2), i.e., a uni-colored tree gives rise to a frame.
We will say that a configuration (resp. shape ) contains a pseudo-frame if are in general position and the corresponding graph to this configuration (resp. shape) has the tree as a subgraph. We conclude from Proposition 4 that every free shape contains a pseudo-frame.
Since pseudo-frames are a generalization of frames, we obtain a topological Hausdorff subspace when considering all shapes containing a fixed pseudo-frame, thus generalizing the definition of and Lemma 3: denote the number of edges in the tree labeled with the landmark by and define .
The topological subspace of all shapes containing a certain pseudo-frame is homeomorphic to the -dimensional differentiable Hausdorff manifold
The final factor of the product in Equation (5) has dimension since is the number of edges in the tree with vertices. This explains the dimension of the manifold.
To show the homeomorphy, consider for a shape (after reordering the rows) a representative of the form in Equation (4). Obviously, the rows of which are not used for the graph give us the first factor of the product in Equation (5). By rescaling of rows and columns the non-zero entries determined by the labeled tree are w.l.o.g. equal to 1, and the rest of the row may be filled with any real number, hence we obtain for row if
Now, Proposition 4 gives us finitely many, manifold-valued charts for whence it is a differentiable manifold.
is a -dimensional differentiable T1 manifold.
From Proposition 4 we obtain homeomorphisms from open subsets of to a differentiable manifold. When composing those with charts of the differentiable manifold, we obtain charts on whose domains cover the full space. Since the transition maps between these charts are just multiplications with non-singular diagonal and non-singular matrices depending smoothly on the representation matrix, the manifold is indeed differentiable.
We would like to point out that for the concept of pseudo-frames adds no extra insight, since a pseudo-frame is already a frame in this case (any colored tree with vertices is uni-colored). For any shape with a pseudo-frame already contains a frame, i.e. for . The critical shape to consider in the case is (in the form of Equation (4))
Let w.l.o.g. there be a pseudo-frame in the first 5 rows of . If either all of or all of , then contains a frame. So, let there be a vanishing value in both of the rows. Since there is a pseudo-frame in the first 5 rows of , there is at most one vanishing value in each row, and it cannot be in the same column. For the sake of argument, let . Then, is a frame. Thus, charts stemming from frames suffice to cover while pseudo-frames give a larger atlas on .
From Theorem 4 follows that free shapes are unblurry which is the converse direction of Proposition 4: is open in since is open in and is an open map. Hence, neighborhoods of in are already neighborhoods of in . Now, is T1 by Theorem 4 whence the intersection of all neighborhoods of in is just , so is the intersection of all neighborhoods of in Hence,
Unfortunately, the manifold of the free is never Hausdorff for and . Even the subset is never Hausdorff for any and which will follow from Proposition 5. Note, however, that all open subsets of are differentiable manifolds by Theorem 4. In particular, all topological subspaces of respecting the hierarchy of projective subspace constraints are differentiable manifolds, since then any configuration has an open neighborhood with whence is open in and is open in
The situation in similarity resp. affine shape space is not as complicated [6, 10]: in both cases, the full shape space is not T1. The largest T1 space in similarity shape space is the full space without the trivial shape, while in affine shape space it is the subspace of the free just like in projective shape space. In both cases, the subspace of the free is a differentiable manifold and, in contrast to the projective situation, Hausdorff.
5 Hausdorff subsets
In applications, one is often interested in metric comparisons of different shapes. Therefore, the underlying shape space needs to be a metrizable topological space (e.g. a Riemannian manifold) which is—of course—at least Hausdorff. Hence, we are looking for topological Hausdorff subspaces of projective shape space.
Consider a shape which fulfills the projective subspace constraint , which may be trivial or non-trivial, i.e., has a representative
for some matrices , and ( possibly being zero). Additionally, consider the sequence with
for some . This sequence converges to and to with
as goes to infinity. But for some choices for as may break some projective subspace constraint. Hence, a topological subspace of containing such and for all would not be Hausdorff since sequences in Hausdorff spaces have at most one limit point. Note that fulfills the projective subspace constraint
This observation can be strengthened to the following result for determining if a projective subspace of is Hausdorff.
Let be a topological subspace which contains and is not Hausdorff. Then, there are two shapes with and . More precisely, is not Hausdorff if and only if there are two distinct shapes which after simultaneous reordering of rows have the form
where are matrices of the same dimensions, and
if , then with diagonal and non-singular, non-singular,
if there is a pair with and ,
if there is a pair with and .
Note that columns can be reordered by the right-action of . The form of the matrices and is illustrated in Figure 3.
The strategy of the proof is as follows: first, we will show that a topological non-Hausdorff subspace contains two shapes of the described form. This will be demonstrated by using the definition of Hausdorff spaces via sequences in first-countable spaces: if with a first-countable topological space do not possess disjoint open neighborhoods, then there is a sequence with limit points and . In shape space, this gives us the sequences with for all and distinct limit points . We will show that w.l.o.g. is diagonal for all and that the sequences converge to singular matrices. Different speeds of convergence lead to the described form of the limit points.
For the other direction, we will again use the idea of different speeds of convergence to construct, like in the proof of Proposition 4, a shape in any neighborhood of some of the described form.
Now, let with such that there are no disjoint open neighborhoods of and Since the topology of is determined by sequences, there is a sequence in with limits W.l.o.g. for all since is dense in and contained in . Thus, there are sequences with limit and with limit in the configuration space such that