An Application of Manifold Learning in Global Shape Descriptors

01/08/2019
by   Fereshteh S. Bashiri, et al.
0

With the rapid expansion of applied 3D computational vision, shape descriptors have become increasingly important for a wide variety of applications and objects from molecules to planets. Appropriate shape descriptors are critical for accurate (and efficient) shape retrieval and 3D model classification. Several spectral-based shape descriptors have been introduced by solving various physical equations over a 3D surface model. In this paper, for the first time, we incorporate a specific group of techniques in statistics and machine learning, known as manifold learning, to develop a global shape descriptor in the computer graphics domain. The proposed descriptor utilizes the Laplacian Eigenmap technique in which the Laplacian eigenvalue problem is discretized using an exponential weighting scheme. As a result, our descriptor eliminates the limitations tied to the existing spectral descriptors, namely dependency on triangular mesh representation and high intra-class quality of 3D models. We also present a straightforward normalization method to obtain a scale-invariant descriptor. The extensive experiments performed in this study show that the present contribution provides a highly discriminative and robust shape descriptor under the presence of a high level of noise, random scale variations, and low sampling rate, in addition to the known isometric-invariance property of the Laplace-Beltrami operator. The proposed method significantly outperforms state-of-the-art algorithms on several non-rigid shape retrieval benchmarks.

READ FULL TEXT VIEW PDF

Authors

page 18

10/23/2011

Spectral descriptors for deformable shapes

Informative and discriminative feature descriptors play a fundamental ro...
10/21/2018

Learning Spectral Transform Network on 3D Surface for Non-rigid Shape Analysis

Designing a network on 3D surface for non-rigid shape analysis is a chal...
10/17/2017

Deep Spectral Descriptors: Learning the point-wise correspondence metric via Siamese deep neural networks

A robust and informative local shape descriptor plays an important role ...
10/02/2019

Model Order Reduction for Efficient Descriptor-Based Shape Analysis

In order to investigate correspondences between 3D shapes, many methods ...
12/25/2018

A Survey on Non-rigid 3D Shape Analysis

Shape is an important physical property of natural and manmade 3D object...
07/30/2019

Bilateral Operators for Functional Maps

A majority of shape correspondence frameworks are based on devising poin...
04/08/2021

Laplace-Beltrami based Multi-Resolution Shape Reconstruction on Subdivision Surfaces

The eigenfunctions of the Laplace-Beltrami operator have widespread appl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Three-dimensional models are ubiquitous data in the form of 3D surface meshes, point clouds, volumetric data, etc. in a wide variety of domains such as material and mechanical engineering [omrani16], genetics [ng2007neuroinformatics], molecular biology [gao2016mesh], entomology [Sosa20163d7743319], and dentistry [riehemann2011microdisplay, wu2016model]

, to name a few. Processing such large datasets (e.g., shape retrieval, matching, or recognition) is computationally expensive and memory intensive. For example, to query against a large database of 3D models to find the closest match for a 3D model of interest, one needs to develop an appropriate similarity measure as well as an efficient algorithm for search and retrieval. Shape descriptors assist with the example problem by providing discriminating feature vectors for shape retrieval 

[aflalo2011deformable, bronstein2011shape] and play a fundamental role when dealing with shape analysis problems such as shape matching [xie2015deepshape, toldo2009visual] and classification [bu2014learning].

In general, there are two types of shape descriptors: local descriptors, also called point signatures, and global descriptors, referred to as shape fingerprints. A local shape descriptor computes a feature vector for every point of a 3D model. On the other hand, a global shape descriptor represents the whole 3D shape model in the form of a low-dimension vector. A descriptor that is informative and concise captures as much information as possible from the 3D shape including the geometric and topological features. Such a vector drastically lowers the shape analysis burdens in terms of both computational intensity and memory.

While a large number of successful non-spectral shape descriptors have been proposed in the literature, spectral descriptors have proved to be beneficial in many applications [boscaini2016anisotropic, bronstein2011spectral]

. The spectral methods take advantage of eigen-decomposition of the Laplace-Beltrami (LB) operator applied on the shapes and construct their informative descriptors using the eigenvalues and eigenvectors. These methods have found successful applications in graph processing 

[raviv2013graph], computational biology [de2013isometry], and point-to-point correspondence [ovsjanikov2012functional].

One of the first spectral descriptors introduced to the computer graphics community is Shape-DNA, developed by Reuter et al. in 2006 [reuter2006laplace]. Shape-DNA attracted a great deal of attention for its unique isometric and rotation invariant features [reuter2006laplace]. Since then, several local as well as global shape descriptors have been introduced in accordance with Shape-DNA such as Heat Kernel Signature (HKS) [sun2009concise], Wave Kernel Signature (WKS) [aubry2011wave], and Global Point Signature (GPS) [rustamov2007laplace]. The common ground between these methods is the discretization approach used to solve the Laplacian eigenvalue problem, which uses a cotangent weighting scheme along with area normalization.

Although there are many advantages of using variations of the cotangent scheme, there are several limitations. First, by their nature, they are limited to triangulated meshes. Second, they do not perform well when dealing with degenerate and non-uniform sampled meshes [reuter2009discrete, belkin2008discrete]. Also, their convergence error depends on factors such as the linearity of the function on the surface [belkin2008discrete]. One possible approach to address these limitations is through the use of manifold learning.

Nonlinear dimensionality reduction techniques, known as manifold learning, assume the existence of a low-dimensional space, which represents a high-dimensional manifold without much loss of information [goldberg2008manifold]. Similar to global descriptors, manifold learning methods attempt to learn the geometry of a manifold in order to extract a low dimensional vector of features that is informative and discriminative. However, unlike shape descriptors, the number of dimensions of a space does not confine manifold learning methods. To the best of our knowledge, the application of manifold learning, an active research topic in statistics and machine learning, has not been investigated in the computer graphics community for extracting global shape descriptors. This motivates the primary aim of this research, which is to explore the effectiveness of a manifold learning method, more specifically Laplacian Eigenmap [belkin2003laplacian], in representing a 3D model with a low-dimensional vector. Our work introduces a novel Laplacian Eigenmap-based global shape descriptor and provides a straightforward normalization method that significantly outperforms existing state-of-the-art approaches.

In our first contribution, inspired by the idea of Laplacian Eigenmaps [belkin2003laplacian], we learn the manifold of a 3D model and then, analogous to the approach taken by Shape-DNA, we make use of the spectrum of the embedded manifold to build the global shape descriptor. This approach has two main advantages. First, it relies on the adjacency of the nodes, disregarding the fine details of the mesh structure. Therefore, it can be used for degenerate or non-uniform sampled meshes. Second, as manifold learning does not rely on the mesh structure and is not limited to a specific type of meshes, e.g., triangulated meshes, it can be applied easily to any other mesh types such as quadrilateral meshes.

In our second contribution, we present a simple and straightforward normalization technique (motivated by [reuter2006laplace, bronstein2010scale, kuang2015modal]) to obtain a scale-invariant global shape descriptor that is more robust to noise. To this end, we propose to subtract the first non-zero eigenvalue from the shape descriptor after taking the logarithm of the spectrum. One advantage of our approach over the idea of Bronstein et al. [bronstein2010scale] is that we avoid taking the direct derivative; this advantage is significant since the differential operator amplifies the noise. Taking the logarithm additionally helps to suppress the effect of the noise that is present in higher order elements of the spectrum.

The remainder of this paper is organized as follows. In Section 2, we briefly overview spectral shape analysis and manifold learning. Then in Section 3, we introduce the proposed shape descriptor along with some technical background. In Section 4, the performance of the proposed method, as well as the robustness of the algorithm are examined and compared with multiple well-known shape descriptors by performing several qualitative and quantitative experiments using widely used 3D model datasets. Section 5 discusses the results in more detail and draws conclusions.

2 Background

In this section, we first review spectral shape analysis, more specifically global shape fingerprints, and different discretization methods of the LB operator. Then, we briefly review manifold learning, more specifically Laplacian Eigenmap, to provide the necessary foundation for developing our proposed Laplacian Eigenmap based scale-invariant shape descriptor, which from now on we call LESI.

2.1 Spectral Shape Analysis

The Laplace-Beltrami operator is a linear differential operator defined on the differentiable manifold as the divergence of the gradient of a function as the following form [reuter2006laplace, gao2014compact]:

(1)

Lévy [levy2006laplace]

noted that the eigenfunctions

of the continuous LB operator, which are the solution to the following Laplacian eigenvalue problem:

(2)

are the orthogonal basis for the space of functions defined on the surface of a manifold. In other words, a function on the surface can be expressed as a sum over coefficients of these infinite bases:

Furthermore, the LB operator is positive semi-definite, having non-negative eigenvalues that can be sorted as follows:

The sequence of eigenvalues of the LB operator is called the spectrum of the LB operator. As it is computed based on the gradient and divergence that depend on the Riemannian structure of the manifold, it possesses the isometry invariant property [reuter2006laplace].

These significant features of the LB operator, which include the orthogonal basis and non-negative spectrum, motivated researchers to develop various local and global shape descriptors. The Shape-DNA and HKS were developed by considering the heat distribution as the function on the surface. The WKS was obtained by solving the Schrödinger wave equation on the surface of the manifold. Also, it has been shown that the GPS descriptor is in close relation to the Green’s function on the surface [rustamov2007laplace].

To approximate equation (2), despite the choice of function , we need a discretization scheme. Different discretization schemas (e.g., Taubin [taubin1995signal], Mayer [mayer2001numerical]) of the LB operator on the triangular meshes are discussed in [xu2004discrete].

A 3D shape, sampled from the surface of a Riemannian manifold , is usually presented by a set of vertices and their connectivity in the form of the graph . For a surface mesh , according to [reuter2009discrete], the equation (2) can be discretized as:

(3)

where is the stiffness matrix and is the lumped mass matrix. One popular approach to constructing the matrix , is using weights:

(4)

where and are the two angles facing the edge . Different mass normalization methods using the triangle area or the Voronoi region area are suggested to construct the matrix . The cotangent weighting schema and its variants have been utilized in multiple FEM-based discretization methods.

Another approach to constructing the matrix is to use the heat kernel weight, also known as the exponential weighting scheme, as follows:

(5)

where denotes the Euclidean distance between two adjacent nodes and .

In [reuter2009discrete], several existing discretization methods, including variants of linear FEM [desbrun1999implicit, meyer2003discrete] and heat kernel weighting proposed in [belkin2008discrete] are compared. According to [reuter2009discrete] and from the discussion led by Xu in [xu2004discrete, xu2006convergence], discrete LB operator using cotangent weighting scheme may not converge in all situations, specifically when dealing with non-uniform meshes. However, the heat kernel weighting scheme proposed in [belkin2008discrete] does not depend on the peculiarities of the triangulation and outperforms all linear approaches [reuter2009discrete, sun2009concise]. In addition, concerning the type of the function , the cotangent scheme only converges for linear functions, while the heat kernel scheme converges well for nonlinear functions as well [belkin2008discrete]. The proposed exponential approximation scheme provides point-wise convergence with good stability with respect to noise. It is important to note that although the method was discussed for surfaces without boundary, the results are valid for interior points of a surface with boundary [belkin2008discrete].

2.2 Manifold Learning

To make the current contribution self-contained, we provide a brief introduction from the data analysis perspective. Dimensionality reduction of high-dimensional data is a critical step in data analysis and processing. Non-linear dimensionality reduction, also known as

manifold learning, is a problem of finding a low-dimensional representation for high-dimensional data. Several local and global manifold learning methods have been developed including Isomap [tenenbaum2000global, silva2003global], LLE [roweis2000nonlinear, saul2003think], Laplacian Eigenmap [belkin2003laplacian], and Diffusion maps [coifman2006diffusion]. We review these next.

Consider a set of points on manifold embedded in . Manifold learning methods look for a set of corresponding points in as a structural representation, while respecting some local or global information. Each method attempts to minimize a cost function in this mapping.

Laplacian Eigenmap [belkin2002laplacian], proposed by Belkin and Niyogy in 2002, is a computationally efficient and mathematically well-studied manifold learning technique. It is based upon graph Laplacian and Laplace-Beltrami operator on the manifold. Accordingly, it is considered as a spectral analysis method. Laplacian Eigenmap deals with sparse, symmetric, and positive semi-definite matrices. It is in close connection to the heat flow [belkin2002laplacian, belkin2003laplacian].

Briefly speaking, for a given manifold, Laplacian Eigenmap applies the graph Laplacian operator and uses the eigenfunctions of such operator to provide the optimal embedding. Laplacian Eigenmap preserves local information by minimizing the distance between embedded points, which are mapped from adjacent points in the original high-dimensional space [belkin2003laplacian]. Aside from the locality preserving property, it provides structural equivalence and discrimination by capturing the intrinsic geometric structure of the manifold. The structural equivalence property states that two similar manifolds will have similar representation after projecting into a lower dimension space [wachinger2010manifold, wachinger2010structural].

Some other manifold learning methods, e.g., Isomap, LLE, and Diffusion map are also based on spectral analysis of the high-dimensional manifold. In contrast to these methods that construct the orthogonal basis of their desired low-dimensional space using eigenfunctions of an LB operator, we develop our scale-invariant shape descriptor using the spectrum of the LB operator. Even though our primary focus is on the Laplacian Eigenmap, owing to its unique properties, we believe that other spectral manifold learning methods are also capable of extracting informative and discriminative shape fingerprints.

3 Method

In this section, we elaborate our proposed LESI global shape descriptor. A flowchart of the proposed approach is shown in Figure 1.

Construct: Laplacian matrix ()

Solve:

Spectrum:

Normalization:

3D model
Figure 1: The block diagram of the proposed Laplacian Eigenmaps based scale-invariant (LESI) global shape descriptor.

3.1 Laplacian Eigenmap-Based Shape Descriptor

We treat a global descriptor as a dimensionality reduction problem as it squeezes the latent information of a 3D model into a vector. Due to the fact that the Laplacian Eigenmap has two properties of structural equivalence and locality preservation [belkin2003laplacian], we propose a global shape descriptor using the spectrum of graph Laplacian.

A graph Laplacian is constructed over an undirected weighted graph with a set of points and a set of edges that connects nearby points (). The theory behind finding the optimal embedding in a Laplacian Eigenmap requires an undirected graph. Every 3D model is given in bidirectional connection and hence, we need to neither examine nor force it to the graph. However, as we will explain later, we need to remove isolated points. Considering the advantages of the heat kernel weighting scheme, which are summarized in Section 2.1 and discussed in details in [belkin2008discrete], Laplacian Eigenmap suggests constructing the weighted graph as follows:

(6)

The only parameter in equation (6) is , which defines the extent to which distant neighbors influence the embedding of each point. The choice of parameter is data-dependent, and there exists no unique way in the literature to select the proper value, but it can be tuned empirically. As has neither a very high impact on the final embedding nor the convergence rate of our final derivations, we empirically recommend:

(7)

Here, weights are bounded as .

Laplacian Eigenmap attempts to find a low dimensional data set that preserves local information. For this purpose, it assumes two neighboring points and stay close after being mapped to and . Therefore, it minimizes the following function [belkin2003laplacian]:

(8)

where is the so called Laplacian matrix, is a symmetric weight matrix, and , the degree matrix, is a diagonal matrix. The assumption that the graph is undirected yields the symmetric property of and consequently, and . It plays a critical role in deriving equation (8).

By adding the orthogonality constraint in order to eliminate the trivial solution and the constraint for removing an arbitrary scaling factor in the embedding, the minimization problem (8) simplifies to:

(9)

The matrix is real, symmetric, and positive semi-definite. Therefore, the solution vector y (in equation (9)) is obtained by the minimum eigenvalue solution to the generalized eigenvalue problem [belkin2003laplacian]:

(10)

At this point, the optimal low dimensional embedding, suggested by the Laplacian Eigenmap, is obtained by utilizing the eigenvectors. However, we focus on the spectrum of the graph Laplacian and its’ properties. Eigenvalues obtained from equation (10) are real, non-negative, and sorted in increasing order as follows:

As the row (or column) sum of is equal to zero, eigenvalue and a corresponding eigenvector 1 are trivial solutions to equation (10). The multiplicity of eigenvalue zero is associated with the number of connected components of the graph. Eigen-solvers often obtain very small, though not precisely zero, eigenvalues due to the computational approximations. If we may know the number of connected components of , we can discard all eigenvalues equal to zero, and form our shape fingerprint using the more informative section of the spectrum. This is easily done by Dulmage-Mendelsohn decomposition [dulmage1958coverings].

The second smallest eigenvalue, also known as the Fiedler value, is a measure of the connectivity within the graph. If the graph has connected components, our proposed shape descriptor is a set of eigenvalues as:

(11)

The LESI descriptor is composed of the spectrum of the LB operator, and hence, it is isometric invariant, independent of the shape location, and informative. The latter, discussed in spectral graph theory, states that the spectrum of graph Laplacian contains a considerable amount of geometrical and topological information of the graph. Moreover, LESI has the similarity property, caused by the structural equivalence property of Laplacian Eigenmap, meaning that two 3D models from the same class of models have similar fingerprints. Unlike Shape-DNA and other shape descriptors that are based on the cotangent weighting scheme, LESI is not limited to triangulated mesh structures because the Laplacian Eigenmap is capable of dealing with high-dimensional data. For some applications in which scale is not a determinant factor, it is favorable to have a scale-invariant descriptor. A fast and efficient normalization method is presented in Section 3.2.

One important matter to consider is the convergence and accuracy of the proposed fingerprint, which ultimately depends on the heat kernel-based discretization of the LB operator. The cotangent weighting scheme and its variants are sensitive to the peculiarities and quality of the particular triangulation of the mesh (refer to Section 2.1). While an exponential weighting scheme has shown accurate performance in dealing with nonlinear functions over the surface and non-uniform mesh representations, it is not clear how this method can handle manifolds with boundaries [reuter2009discrete, belkin2008discrete]. It does, however, behave well for interior points of the surface. Therefore, we recommend removing rows and columns of and corresponding to isolated points, before solving equation (10). The descriptor obtained from the rest of the connected graph is an informative and discriminative descriptor of the graph.

3.2 Scale Normalization

For some applications, the size of an object is not a determinant factor in shape comparison and identification. Therefore, a scale-invariant shape descriptor with a solid normalization method is more desirable. For that purpose, some shape descriptors including Shape-DNA, have proposed multiple normalization methods. Most normalization methods of Shape-DNA focus on finding an appropriate scaling factor, such as the surface area, the volume, or coefficient of a fitting curve, which will be multiplied in the descriptor.

Moreover, it is shown that eigenvalues with a higher order are more susceptible to noise. For that reason, the original Shape-DNA recommends cropping the spectrum and using no more than 100 eigenvalues [reuter2006laplace]. In this section, we propose a simple and efficient normalization method that significantly reduces the effect of scale variations as well as noise. In this approach, we are interested in taking the scaling factor out from the descriptor in one step, rather than an extra step to find an appropriate neutralizing factor. Although the normalization seems simple, later in the experiments section, we will show its efficiency. The work presented in [sun2009concise] and [reuter2006laplace] influenced this method.

(a) Teddy Bear models
(b) Spectrum of LB operator
(c) Logarithm of spectrum
(d) Normalized Spectrum
Figure 2: An example showing the proposed normalization method of the shape descriptor. LABEL:sub@subfig:teddymodel The Teddy Bear model and its scaled version (scale factor 0.7). LABEL:sub@subfig:spectrum The spectrum of original (blue) and scaled (red) Teddy Bear models. Please note that the original spectrum is approximately multiplied by half. LABEL:sub@subfig:logspectrum The Logarithm of the spectrum shown in LABEL:sub@subfig:spectrum. LABEL:sub@subfig:normspectrum The normalized spectrum of original and scaled Teddy Bear models after subtracting first element of logarithm of the spectrum .

According to Weyl’s law [weyl1911asymptotische], when an arbitrary scale object is scaled with factor , the eigenvalues are scaled with factor . Let be a LESI descriptor obtained from a scaled 3D model with unknown factor . To normalize the descriptor and eliminate the effect of the scaling factor, we recommend computing:

(12)

To achieve a scale invariant shape descriptor, we first take the logarithm of the descriptor vector and then compute the difference of the new vector from its smallest element. This is equivalent to dividing the vector by its first element and taking the logarithm next. Basically, division takes the factor out, and the logarithm eliminates the influence of noise.

Figure 2 illustrates the details of the proposed algorithm in an example. In Figure 2LABEL:sub@subfig:teddymodel, two Teddy Bear models are shown. One model is in the original size whereas the other model is scaled with factor 0.7. It is clear from Figure 2LABEL:sub@subfig:spectrum that the spectrum of the scaled model is almost of the spectrum of the original model. Taking the logarithm of the spectrum takes away the scaling factor from multiplicand and leaves it as augend. Therefore, subtracting one term (e.g., the first element) removes the scaling factor from all other terms. The result is a normalized and scale-free spectrum.

3.3 Algorithm

Our proposed descriptor consists of three major steps. For a given 3D polygonal model with a set of vertices and a set of neighbor connections , the LESI descriptor is a -dimensional vector of real and positive values.

In the first step, we compute the real, symmetric, and sparse weight matrix for a 1-ring neighbor of every point as stated in equation (6) using the inner scaling factor given in equation (7). Next, we form the generalized eigenvalue problem equation (10) by constructing Laplacian and degree matrices ( and respectively) without difficulty. The matrix is sparse, real, symmetric and semi-positive. Utilizing the Dulmage-Mendelsohn decomposition, we find the number of connected components of . The objective of the second step is to find the spectrum of the LB operator. For that purpose, we solve the generalized eigenvalue problem using the Lanczos method. Then, we leave out as many smallest eigenvalues as the number of connected components. Since in most cases a single 3D model is made up of one connected component, we only need to leave out one eigenvalue. The last step of the algorithm deals with scale normalization and noise reduction, in case it is required, by taking the logarithm of the spectrum and subtracting the first element from the rest of the vector. Detailed steps of the algorithm are summarized in Algorithm 1.

Input: A 3D polygonal model with n vertices and edge list
Output: A -dimensional vector
1 Compute edge weights using equations (6) and (7);
2 Construct the sparse, real, and symmetric matrices , , and ;
3 Find number of connected components (nConComp) from ;
4 Solve equation (10) for eigenvalues;
5 Sort them in increasing order and leave out smallest ones;
6 if normalization is required then
7       Compute where
8      ;
9      
10 end if
Algorithm 1 Laplacian Eigenmap-based scale-invariant global shape descriptor

4 Experiments

In Section 4.1, we first present two datasets used in our experiments. Then, in Section 4.2, we qualitatively visualize and measure the competence of the proposed method in discriminating different clusters compared with candidate methods from the literature. Next, in Section 4.3, we validate the effectiveness of the LESI descriptor to distinct multiple classes by measuring the accuracy of multi-class classification. Finally, in Section 4.4, extensive experiments are carried out to study the robustness of the proposed shape descriptor with respect to noise, scale invariance, and down-sampling. All the algorithms were implemented using the MATLAB R2013a environment running on a personal computer with Intel(R) Core(TM) i3-4130 CPU @ 3.40GHz and 12GB memory.

4.1 Dataset

To validate the utility of the proposed shape descriptor, we utilized two standard, widely-used, and publicly available datasets of 3D polygon meshes. The high-resolution TOSCA dataset [bronstein2008numerical] contains 80 three-dimensional non-rigid models, including 11 cats, 6 centaurs, 9 dogs, 4 gorillas, 8 horses, 12 women poses, 3 wolves and two men with 7 and 20 poses respectively. In our experiments, we use all models except the gorilla models, as they contain isolated points. The models in each class of the TOSCA dataset are almost identical in terms of scale, the number of vertices, quality of triangulation, and structure, which all represent the same object with different poses.

The McGill dataset with articulating parts [siddiqi2008retrieving] is used to evaluate the ability of the descriptors to describe models with poor intra-class quality. The McGill dataset contains 3D models of 30 ants, 30 crabs, 25 glasses, 20 hands, 29 humans, 25 octopuses, 20 pliers, 25 snakes, 31 spiders, and 20 Teddy bears. The classification of the McGill dataset models is more challenging due to scale and shape variations.

4.2 Retrieval Results

In this section, we evaluate the general performance of our proposed shape descriptor and compare it with several state-of-the-art spectral-based global shape descriptors including Shape-DNA [reuter2006laplace], cShape-DNA [gao2014compact], and GPS [rustamov2007laplace] algorithms. We chose these methods because they are widely used by researchers (e.g., [Mirloo2017, masoumi2017global, boscaini2015learning]) to develop new descriptors or applications, or to evaluate the performance of their proposed descriptors. Moreover, cShape-DNA represents the normalized version of the original Shape-DNA. Even though there are multiple ways to convert a local point descriptor to a global shape fingerprint, in this article we focus only on algorithms that have been originally introduced as global fingerprints. To this end, we take advantage of the source code made available on Dr. Kokkinos’s homepage111http://vision.mas.ecp.fr/Personnel/iasonas/descriptors.html [bronstein2010scale], as well as the shape descriptor package provided by Li et al. [li2014spatially] available on a GitHub repository222https://github.com/ChunyuanLI/spectral_descriptors to generate the Shape-DNA and GPS descriptors, respectively. We also compare the performance of shape retrieval using the code provided for evaluation by SHREC’11 [lian2011shape].

The shape descriptors are compared using the TOSCA dataset to discriminate between different classes of 3D objects. In this experiment, we use the first 33 non-zero eigenvalues . Then, to visualize the locations of objects in the shape space, we project them onto a 2D plane using Principle Component Analysis (PCA). Figure 3 displays the effectiveness of our method compared with the fingerprints of interest.

(a) Shape-DNA
(b) cShape-DNA
(c) GPS
(d) LESI
(e)
Figure 3: 2D PCA projection of shape descriptors computed from LABEL:sub@subfig:tosca-shapedna-2dmds original Shape-DNA, LABEL:sub@subfig:tosca-cshapedna-2dmds cShape-DNA, LABEL:sub@subfig:tosca-gps-2dmds GPS, and LABEL:sub@subfig:tosca-lesi-2dmds LESI algorithms on TOSCA dataset.

Figure 3 reveals that LESI can differentiate models of various classes significantly better than the other methods for a refined and normalized dataset. Even though all human models (David, Michael, and Victoria) are very similar, it can distinguish the women from the men’s group. However, it fails to discriminate models of Michael from David. Despite the large isometric deformations in each class, the proposed LESI method clusters all models of the same class together very tightly.

To demonstrate the power of our method in classifying objects with low intra-class similarity compared with other shape descriptors, the same experiment is carried out on the McGill dataset. Models of the same class with articulating parts are in different scales, shape, and structure. The 2D PCA projections of 33-dimension descriptors from all four algorithms are shown in Figure 

4.

(a) Shape-DNA
(b) cShape-DNA
(c) GPS
(d) LESI
(e)
Figure 4: 2D PCA projection of shape descriptors computed from LABEL:sub@subfig:mcgill-shapedna-2dmds original Shape-DNA, LABEL:sub@subfig:mcgill-cshapedna-2dmds cShape-DNA, LABEL:sub@subfig:mcgill-gps-2dmds GPS, and LABEL:sub@subfig:mcgill-lesi-2dmds LESI algorithms on McGill dataset.

As illustrated in Figure 4, the original Shape-DNA is highly sensitive to scales. Multiple methods are presented in [reuter2006laplace] to make the descriptor normalized to scale. cShape-DNA represents a normalized version of it by multiplying the descriptor with the surface area. Although cShape-DNA can separate models from each other, classes are not separated efficiently. LESI outperforms the other algorithms by providing distinct descriptors, which can separate classes. Shape descriptors offered by LESI prove superior to the other algorithms in the shape retrieval and classification tasks, as described below and in the next section respectively.

Dataset Method NN FT ST E DCG
TOSCA ShapeDNA 1.0000 0.8091 0.9391 0.4486 0.9584
cShapeDNA 0.9474 0.7748 0.8984 0.4748 0.9241
GPS 0.4868 0.4244 0.6320 0.3614 0.6787
LESI 0.8684 0.8456 0.9430 0.4860 0.9244
McGill ShapeDNA 0.7922 0.3452 0.4977 0.3411 0.7192
cShapeDNA 0.7882 0.3943 0.5483 0.3852 0.7470
GPS 0.3843 0.2508 0.4066 0.2588 0.6020
LESI 0.9647 0.7046 0.8739 0.6644 0.9251
Table 1: Shape retrieval performance using TOSCA and McGill datasets

To examine the superiority of LESI quantitatively, we computed multiple standard retrieval measures including Nearest Neighbor (NN), First Tier (FT), Second Tier (ST), e-Measure (E), and Discounted Cumulative Gain (DCG). These measures represent state-of-the-art quality metrics used when evaluating matching results for shape-based search engines [shilane2004princeton]. Table (1) reports the results of shape retrieval. Boldface numbers indicate the highest value for each measure per each dataset. From Table (1), it is clear that the LESI descriptor outperforms all other methods concerning all measures in retrieving models from the McGill dataset. When retrieving models of the TOSCA dataset, LESI outperforms all methods concerning FT, ST, and E measures. Shape-DNA outperforms LESI by a higher value for NN and DCG measures, due to the poor discrimination between David and Michael performed by the LESI descriptor. However, it does not diminish the validity of our claim that LESI performs well for meshes with non-uniform sampling or peculiarities.

4.3 Multi-class Classification Results

In this section, we corroborate the findings of Section 4.2 by training a linear multi-class SVM classifier to assess the accuracy of LESI compared to other shape descriptors. For this experiment, we utilized the McGill dataset. In addition to the shape descriptors evaluated in Section 4.2, we computed another normalized version of Shape-DNA by dividing the feature vector by its first element (similar to what LESI offers) as suggested in [reuter2006laplace]. This way we can compare the effect of the exponential weighting scheme without the influence of the normalization method or compactness (offered by cShape-DNA). Using 10-fold cross-validation and repeating the experiment 3 times, we report the average accuracy for each method in Table (2).

The new LESI approach significantly outperforms all other methods when using a two-tailed paired t-test

. The t-test was performed on one set of 10 folds in order to avoid violating the independence assumption of the t-test. There is a significant improvement in accuracy when comparing the Shape-DNA (Normalized) to other variants of the Shape-DNA, which is due in part to the normalization method. However, the average accuracy of the LESI descriptor is noticeably higher (95%) when compared to 90% of the Shape-DNA (Normalized).

Method Average accuracy
Shape-DNA 21.02%
Shape-DNA (Normalized) 90.60%
cShape-DNA 71.37%
GPS 50.11%
LESI 95.69%
Table 2: Classification accuracy using McGill dataset
Figure 5: Confusion matrix obtained from linear multi-class SVM for McGill dataset using LESI descriptors.

Finally, Figure 5 shows the confusion matrix obtained from the linear multi-class SVM using LESI descriptor. The number of correct classifications made for each class (indicated by the green diagonal), confirms that our method captures the discriminative features of the shapes.

4.4 Robustness

In this section, we address the robustness of the LESI shape descriptor to shape variations, including noise, scale, and down-sampling by performing another set of experiments. First, we generate the disturbed version of every model in the TOSCA dataset. Then, we test the capability of every method mentioned above in discriminating between different classes. For this purpose, besides plotting the 2D PCA projection of shape descriptors, we also compute and plot the pairwise Euclidean distance matrix, in every case. The distance matrix represents the dissimilarity between each pair of models in the set. It is often used to compute other evaluating metrics such as nearest-neighbor, and first and second tier, to name a few. The dissimilarity of descriptors increases from blue to red, and the more separate classes differ in color, the better they are discriminating from each other.

Resistance to noise. Multiple noisy versions of the TOSCA dataset are generated following the idea articulated in [liu2011construction]. To this end, the surface meshes of all models are disturbed by changing the position of each point along its normal vector that is chosen randomly from an interval with the 0 mean, where determines the noise level and is a fraction of the diagonal length of the model bounding box. In this experiment, three noise levels , , and are tested, where the latter one represents a greater level of noise. Two-dimensional PCA projections of all descriptors with the presence of different levels of noise are plotted in Figure 6. Combining these with the results shown in Figure 3, where no noise is present, demonstrates that the LESI algorithm is highly noise-resistant while the performance of the Shape-DNA and cShape-DNA decreases as the level of noise increases. Moreover, GPS fails in separating different classes of models with the presence of noise. Figure 9 reflects the effect of noise on the discriminative power of the descriptors. The LESI algorithm shows consistent results as the level of noise increases from 0% (top row) to 2% (bottom row).

(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
(m)
(n)
(o)
Figure 6: 2D PCA projection of shape descriptors computed by (from left to right) Shape-DNA, cShape-DNA, GPS, and LESI algorithms from perturbed TOSCA dataset with (from top to bottom) 0.5%, 1%, 2% noise level, respectively.

Scale invariance. In order to validate the insensitivity of the LESI descriptor to scale variations and compare the robustness of the proposed method with other descriptors, each model of the TOSCA dataset is scaled by a factor of 0.5, 0.875, 1.25, 1.625, or 2 randomly. Figure 7 shows that the LESI algorithm surpasses other methods in discerning different classes. Comparing the result of this experiment with the results shown in Figure 3 demonstrates the consistency of the LESI algorithm with the presence of scale variation. The distance matrices in Figure 10 show that the original Shape-DNA algorithm is very susceptible to scale variations. Even though the cShape-DNA has significantly improved scale sensitivity of the original Shape-DNA, it does not provide as accurate results as the LESI algorithm does.

(a) Shape-DNA
(b) cShape-DNA
(c) GPS
(d) LESI
(e)
Figure 7: 2D PCA projection of shape descriptors computed by LABEL:sub@subfig:tosca-rndscale-shapedna-2dmds original Shape-DNA, LABEL:sub@subfig:tosca-rndscale-cshapedna-2dmds cShape-DNA, LABEL:sub@subfig:tosca-rndscale-gps-2dmds GPS, and LABEL:sub@subfig:tosca-rndscale-lesi-2dmds LESI algorithms over scaled version of the TOSCA dataset by a randomly chosen factor of 0.5, 0.875, 1.25, 1.625, or 2.
(a) Shape-DNA
(b) cShape-DNA
(c) GPS
(d) LESI
(e)
Figure 8: 2D PCA projection of shape descriptors computed by LABEL:sub@subfig:tosca-dwnsmpl20-shapedna-2dmds original Shape-DNA, LABEL:sub@subfig:tosca-dwnsmpl20-cshapedna-2dmds cShape-DNA, LABEL:sub@subfig:tosca-dwnsmpl20-gps-2dmds GPS, and LABEL:sub@subfig:tosca-dwnsmpl20-lesi-2dmds LESI algorithms from down sampled TOSCA dataset by rate of 20%.

Resistance to the sampling rate. To investigate the effect of sampling rates on the discriminative power of the shape descriptors, Bronstein et al. [bronstein2011shape] propose to reduce the number of vertices to 20% of its original size. Accordingly, the down-sampled version of the TOSCA dataset is generated, and shape descriptors associated with them are computed. The 2D PCA projections and distance matrices of descriptors are illustrated in Figures 8 and 11, respectively. Although the original Shape-DNA shows a more accurate result than cShape-DNA, the separation of cat, dog, and wolf models is challenging. Although the performance of the LESI method is slightly affected, it still outperforms cShape-DNA and GPS methods.

Noise
Level
Shape-DNA cShape-DNA GPS LESI
0%
0.5%
1%
2%
Figure 9: The Euclidean pairwise distance matrix of shape descriptors computed by (from left to right) Shape-DNA, cShape-DNA, GPS, and LESI algorithms from perturbed TOSCA dataset by (from top to bottom) 0%, 0.5%, 1%, 2% noise levels.
Shape-DNA cShape-DNA GPS LESI
Random
Scale
Figure 10: The Euclidean pairwise distance matrix of shape descriptors computed by (from left to right) Shape-DNA, cShape-DNA, GPS, and LESI algorithms over scaled version of the TOSCA dataset by a randomly chosen factor of 0.5, 0.875, 1.25, 1.625, or 2.
Shape-DNA cShape-DNA GPS LESI
Down
Sampled
Figure 11: The Euclidean pairwise distance matrix of shape descriptors computed by (from left to right) Shape-DNA, cShape-DNA, GPS, and LESI algorithms from from down sampled TOSCA dataset by rate of 20%.

5 Discussion

In this article, motivated by the unique properties of Laplacian Eigenmap (i.e., locality preservation, structural equivalence, and dimensionality reduction) and inspired by the existing spectral-based shape descriptors, we investigated the application of manifold learning in deriving a shape fingerprint in order to address the limitations tied to popular cotangent-based shape descriptors. We proposed a global descriptor (LESI) with an easy-to-compute and efficient normalization technique that facilitates applications such as shape classification and retrieval. Our method applies fewer restrictions on the class of meshes as well as improving the quality of tessellations. Analogous to other spectral descriptors, LESI uses the spectrum of the LB operator, which is independent of the shape location, is informative (contains a considerable amount of geometrical and topological information), and above all isometric invariant. We compared the discriminating power of LESI with three prominent descriptors from the literature, namely Shape-DNA, cShape-DNA, and GPS, and found it to be superior.

In the first set of experiments illustrated in Figures 3 and 4, our method substantially outperforms the others. The superiority of LESI is more significant when the McGill dataset is used (Table 2 and Figure5). This dataset includes wide variations in mesh structure and scales, causing the failure of the other methods to generate acceptable results. However, LESI, due to utilizing a different method of discretization to form the LB operator, focuses on the vicinity rather than the quality of the triangulation. Therefore, our technique, unlike other methods, is not affected by the low quality of polygon meshes.

The second set of experiments evaluates the reliability of our method in the presence of noise, scale variations, as well as different sampling rates. LESI shows impressive robustness against the first two sets of perturbation. Despite the negative impact of down sampling in LESI descriptor, it continues to show better performance when compared to cShape-DNA and GPS. It should be noted that the result could also be improved by increasing the size of the output vector.

In addition to the discriminating power of the descriptor, degenerate and non-uniform meshes may also cause failure of an algorithm to converge. The cotangent weight-based algorithms were not able to compute the descriptors for 2 shapes from the McGill dataset. GPS also failed to compute descriptors for 6 models of the down sampled TOSCA dataset. However, our technique converges at all times despite the quality of the polygon mesh structure.

Moreover, LESI, unlike cotangent weight-based techniques, is not confined to the triangulated meshes as it disregards the mesh geometry [zhang2009surface]. LESI inherits this property from the capability of manifold learning techniques in coping with high dimensional data. The discretization of the LB operator using cotangent weights on the quadrilateral meshes is not as straightforward as on triangular meshes. To compute the LB operator on a quadrilateral mesh, all rectangles need to be divided into triangles. It could be done easily, however, as for each quad there are two possible triangulations, the result is not unique.

In the original Laplacian Eigenmaps, the high dimension data requires a considerable amount of processing as the list of all connections need to be computed for the dataset. In fact, for each point in the high dimension space, a given number of nearest neighbors need to be extracted which could be challenging and unmanageable. While applying this technique to the 3D meshes, we skip this step as the neighbors are already defined and given in the mesh structure.

This work benefits from the Laplacian Eigenmap technique in a space in which the vicinities are given. LESI takes advantage of simple Laplacian computation, to form the LB operator, which provides concise and informative shape descriptors. Experimental results prove that LESI is more effective compared with the other powerful descriptors.

One limitation of LESI is its inability to separate models of different men (David and Michael). However, it was able to differentiate between the wolf and dog, as well as between women and men.

Although we investigated only the application of Laplacian Eigenmap in introducing a shape descriptor, there are some other spectral-based manifold learning methods, such as Isomap, LLE, and Diffusion map, which have not been examined. This can be considered future work.

6 Conclusion

This work presents LESI, a novel scale-invariant global shape descriptor based on Laplacian Eigenmap that is significantly better when compared to other shape descriptors. We conclude that manifold learning methods can be used to develop new spectral-based shape descriptors to learn the structure of manifolds despite the quality of sampled meshes.

7 Acknowledgment

We acknowledge financial support for Drs. D’Souza, Yu and Ms. Bashiri from GE Healthcare through the UWM Catalyst Grant program. Our sincere thanks goes to Dr. Ahmad P. Tafti and Dr. C. David Page for their expertise and constructive comments. We also acknowledge financial support from the Center for Predictive Computational Phenotyping, supported by the National Institutes of Health Big Data to Knowledge (BD2K) Initiative under Award Number U54 AI117924 and the grant UL1TR002373 from the Clinical and Translational Science Award (CTSA) program of the National Center for Advancing Translational Sciences, NIH.

References