Higher-order Projected Power Iterations for Scalable Multi-Matching

11/26/2018 ∙ by Florian Bernard, et al. ∙ 0

The matching of multiple objects (e.g. shapes or images) is a fundamental problem in vision and graphics. In order to robustly handle ambiguities, noise and repetitive patterns in challenging real-world settings, it is essential to take geometric consistency between points into account. Computationally, the multi-matching problem is difficult. It can be phrased as simultaneously solving multiple (NP-hard) quadratic assignment problems (QAPs) that are coupled via cycle-consistency constraints. The main limitations of existing multi-matching methods are that they either ignore geometric consistency and thus have limited robustness, or they are restricted to small-scale problems due to their (relatively) high computational cost. We address these shortcomings by introducing a Higher-order Projected Power Iteration method, which is (i) efficient and scales to tens of thousands of points, (ii) straightforward to implement, (iii) able to incorporate geometric consistency, and (iv) guarantees cycle-consistent multi-matchings. Experimentally we show that our approach is superior to existing methods.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Establishing correspondences is one of the fundamental problems in vision and graphics, as it is relevant in a wide range of applications, including 3D reconstruction, tracking, recognition, or shape matching. The overall goal of correspondence problems is to identify points in objects (e.g. images, meshes, or graphs) that are semantically similar. While matching points independently of their neighbourhood context is computationally tractable (e.g. via the linear assignment problem (LAP) [9]), such approaches are limited to simple cases without ambiguities or repetitive patterns. In order to resolve ambiguities and avoid mismatches in challenging real-world scenarios, it is crucial to additionally incorporate the geometric context of the points, so that spatial distances between pairs of points are (approximately) preserved by the matching. To this end, higher-order information is commonly integrated into the matching problem formulation, e.g. via the NP-hard quadratic assignment problem (QAP) [26].

Multi-matching, i.e. finding matchings between more than two objects (e.g. an image sequence, a multi-view scene, or a shape collection) plays an important role in various applications, such as video-based tracking, multi-view reconstruction (e.g. for AR/VR content generation) or shape modelling (e.g. for statistical shape models in biomedicine [21]). Computationally, finding valid matchings between more than two objects simultaneously is more difficult compared to matching only a pair of objects. This is because one additionally needs to account for cycle-consistency, which means that compositions of matchings over cycles must be the identity matching—even when ignoring higher-order terms, an analogous multi-matching variant of the linear assignment problem

that accounts for cycle-consistency results in a (non-convex) quadratic optimisation problem over binary variables, which structurally resembles the NP-hard quadratic assignment problem. When additionally considering higher-order terms in order to account for geometric relations between the points, multi-matching problems become even more difficult. For example, a multi-matching version of the quadratic assignment problem either results in a fourth-order polynomial objective function, or in a quadratic objective with additional (non-convex) quadratic

cycle-consistency constraints, both of which are to be optimised over binary variables.

Practical approaches for solving multi-matching problems can be put into two categories: (i) methods that jointly optimise for multi-matchings between all objects (e.g. [22, 47, 40, 43, 6]) and (ii) approaches that first establish matchings between points in each pair of objects independently, and then improve those matchings via a post-processing procedure referred to as permutation synchronisation [34, 11, 55, 39, 29]. The approaches that jointly optimise for multi-matchings either ignore geometric relations between the points [43], or are prohibitively expensive such that they can only cope with small-scale problems (cf. Fig. LABEL:fig:teaser). In contrast, while the synchronisation-based approaches are generally much more scalable (e.g. synchronisation problems with a total number of points in the order of k-k can be solved), they completely ignore geometric relations (higher-order terms) during the synchronisation, and thus achieve limited robustness in ambiguous settings (cf. Fig. LABEL:fig:teaser).

The aim of this work is to provide a scalable solution for multi-matching that addresses the mentioned short-comings of previous approaches. Our main contributions are:

  • We propose a method that jointly optimises for multi-matchings which is efficient and thus applicable to large-scale multi-matching problems.

  • Our method is guaranteed to produce cycle-consistent multi-matchings, while at the same time considering geometric consistency between the points.

  • To this end, we propose a Higher-order Projected Power Iteration (HiPPI) method that can be implemented in a few lines of code.

  • We empirically demonstrate that our method achieves beyond state-of-the-art results on various challenging problems, including large-scale multi-image matching and multi-shape matching.

2 Background & Related Work

In this section we review the most relevant works in the literature, while at the same time providing a summary of the necessary background.

Pairwise Matching:

The linear assignment problem (LAP) [9] can be phrased as

(1)

where is a given matrix that encodes (linear) matching costs between two given objects, and is a permutation matrix that encodes the matching between these objects. The LAP can be solved in polynomial time, e.g. via the Kuhn-Munkres/Hungarian method [30] or the (empirically) more efficient Auction algorithm [7]. The quadratic assignment problem (QAP) [26], which reads

(2)

additionally incorporates pairwise matching costs between two objects that are encoded by the matrix . The QAP is a (strict) generalisation of the LAP, which can be seen by defining : since , we obtain that , where is the Hadamard product. However, in general the QAP is known to be NP-hard [35]. The QAP is a popular formalism for graph matching problems, where the first-order terms (on the diagonal of ) account for node matching costs, and the second-order terms (on the off-diagonal of ) account for edge matching costs. Existing methods that tackle the QAP/graph matching include spectral relaxations [27, 14], linear relaxations [42, 41], convex relaxations [53, 37, 33, 18, 1, 24, 17, 6], path-following methods [52, 54, 23]

, kernel density estimation 

[44], branch-and-bound methods [5] and many more, as described in the survey papers [35, 28]

. Also, tensor-based approaches for higher-order graph matching have been considered

[16, 32].

While the requirement implies bijective matchings, in the case of matching only two objects, the formulations (1) and (2) are general in the sense that they also apply to partial matchings, which can be achieved by incorporating dummy points with suitable costs. However, due to ambiguities with multi-matchings of dummy points, this is not easily possible when considering more than two objects.

Multi-matching:

In contrast to the works of [46], where cycle-consistency has been modelled as soft-constraint within a Bayesian framework for multi-graph matching, in [49, 48] the authors have addressed multi-graph matching in terms of simultaneously solving pairwise graph matching under (hard) cycle-consistency constraints. In [50], the authors have generalised factorised graph matching [54] from matching a pair of graphs to multi-graph matching. Another approach that tackles multi-graph matching is based on a low-rank and sparse matrix decomposition [51]. In [47], a composition-based approach with a cycle-consistency regulariser is employed. In [24], the authors propose a semidefinite programming (SDP) relaxation for multi-graph matching by (i) relaxing cycle-consistency via a semidefinite constraint, and (ii) lifting the permutation matrices to -dimensional matrices. In order to reduce computational costs due to the lifting of the permutation matrices, the authors in [6] propose a lifting-free SDP relaxation for multi-graph matching. In [36], the authors propose a random walk technique for multi-layered multi-graph matching. While there is a wide range of algorithmic approaches for multi-graph matching, the aforementioned approaches have in common that they are computationally expensive and are only applicable to small-scale problems, where the total number of points does not significantly exceed a thousand (e.g.  graphs with nodes each). In contrast, our method scales much better and handles multi-matching problems with more than k points.

In [13], the authors use a two-stage approach with a sparsity-inducing -formulation for multi-shape matching. While the effect of this approach is that only very few multi-matchings are found, our approach obtains significantly more multi-matchings, as we will demonstrate later.

Rather than modelling higher-order relations between points, the recent approach [45] accounts for geometric consistency in 2D multi-image matching problems by imposing a low rank of the (stacked) 2D image coordinates of the feature points. On the one hand, this is based on the (over-simplified) assumption that the 2D images depict a 3D scene under orthographic projections, and on the other hand such an extrinsic approach is not directly applicable to distances on non-Euclidean manifolds (e.g. multi-shape matching with geodesic distances). In contrast, our approach is intrinsic due to the use of pairwise adjacency matrices, and thus can handle general pairwise information independent of the structure of the ambient space.

Synchronisation methods:

Given pairwise matchings between pairs of objects in a collection, synchronisation methods have the purpose of improving the given input matchings. The motivation of permutation synchronisation methods is to achieve cycle-consistency in the set of pairwise matchings:

Definition 1.

(Cycle-consistency)
Let be the set of pairwise matchings in a collection of objects, where each is an element of the set of partial permutation matrices

(3)

The set is said to be cycle-consistent, if for all it holds that:

  1. [label=()]

  2. (identity matching),

  3. (symmetry), and

  4. (transitivity).

For the case of full permutation matrices, i.e. in (3) the inequalities become equalities and , Pachauri et al. [34] have proposed a simple yet effective method to achieve cycle-consistency based on a spectral decomposition of the matrix of pairwise matchings. The authors of [39] provide an analysis of such spectral synchronisations. Earlier works have also considered an iterative refinement strategy to improve pairwise matchings [31].

While the aforementioned synchronisation methods considered the case of full permutations, some authors have also addressed the synchronisation of partial matchings, e.g. based on semidefinite programming [11], alternating direction methods of multipliers [55]

, or a spectral decomposition followed by k-means clustering

[2]. A spectral approach has also been presented in [29], which, however, merely improves given initial pairwise matchings without guaranteeing cycle-consistency.

Rather than explicitly modelling the cubic number of cycle-consistency constraints (cf. Def. 1), most permutation synchronisation methods leverage the fact that cycle-consistency can be characterised by using the notion of universe points, as e.g. in [43]:

Lemma 2.

(Cycle-consistency, universe points)
The set of pairwise matchings is cycle-consistent, if there exists a collection

(4)

such that for each it holds that .

Proof.

We show that conditions (i)-(iii) in Def. 1 are fulfilled. Let be fixed. (i) We have that and imply that , so that . (ii) Moreover, means that . (iii) We have that implies . We can write

(5)
(6)

Here, the -dimensional object-to-universe matching matrix assigns to each of the points of object exactly one of universe points—as such, all points among the objects that are assigned to a given (unique) universe point are said to be in correspondence.

For notational brevity, it is convenient to consider a matrix formulation of Lemma 2. With being the number of points in the -th object and , let be the -dimensional pairwise matching matrix

(7)

and let

(8)

Lemma 2 translates into the requirement that there must be a , such that

(9)

With this matrix notation it becomes also apparent that one can achieve synchronisation by matrix factorisation, such as pursued by the aforementioned spectral approaches [34, 39, 2, 29]. While recently a lot of progress has been made for permutation synchronisation, one of the open problems is how to efficiently integrate higher-order information to model geometric relations between points. We achieve this goal with our proposed method and demonstrate a significant improvement of the matching accuracy due to the additional use of geometric information.

Power method:

The power method is one of the classical routines within numerical linear algebra for computing the eigenvector corresponding to the largest (absolute) eigenvalue. For a given (symmetric) matrix

, the power method iteratively updates , where it can be shown that converges to an eigenvector of  [20]. Moreover, it is well-known that the eigenvector corresponding to the largest (absolute) eigenvalue maximises the Rayleigh quotient , which, up to scale, is equivalent to maximising the (not necessarily convex) quadrative objective over the unit sphere. In addition to computing a single eigenvector, straightforward extensions of the power method are the Orthogonal Iteration and QR Iteration methods [20], which simultaneously compute multiple (orthogonal) eigenvectors. Analogously to the power method, such methods can be used to maximise the (not necessarily convex) quadratic objective over the Stiefel manifold, i.e. under the orthogonality constraint .

Power method generalisations:

In addition to optimising quadratic objective functions, higher-order generalisations of the power method have been proposed. For rank-1 tensor approximation, Lathauwer et al. have proposed the Higher-order Power Method [15]. In [40], the authors propose Tensor Power Iterations for the problem of multi-graph matching. However, their approach has a runtime complexity that is exponential in the number of graphs and thus prevents scalability (e.g. matching graphs, each with nodes, takes about minutes). In [10], the authors propose a Projected Power Method for the optimisation of quadratic functions over sets other than the Stiefel manifold, such as permutation matrices. In a permutation synchronisations setting, their method obtains results that are comparable to semidefinite relaxations methods [11, 22] at a reduced runtime. However, due to the restriction to quadratic objective functions, their approach cannot handle geometric relations between points, as they would become polynomials of degree four, as will be explained in Sec. 3. Our method goes beyond the existing approaches as we propose a method that resembles a projected power iteration for maximising a higher-order objective over the set . With that, we can incorporate geometric information between neighbouring points using a fourth-order polynomial, while always maintaining cycle-consistency.

3 Method

The overall idea of our approach is to phrase the multi-matching problem as simultaneously solving pairwise matching problems that incorporate linear (first-order) and quadratic (second-order) terms. However, instead of directly optimising over pairwise matchings, we parametrise the pairwise matchings in terms of their object-to-universe matchings, cf. Lemma 2 and Eq. (9). While this has the advantages that (i) cycle-consistency is guaranteed to be always maintained, as well as that (ii) one only optimises for (rather than variables in the pairwise case; where commonly ), one disadvantage is that the linear term becomes quadratic, and the quadratic term becomes quartic (a fourth-order polynomial). In the following we will elaborate on this.

3.1 Multi-Matching Formulation

Our multi-matching formulation that optimises over the object-to-universe matchings reads

(10)

Here, encodes the (non-negative) similarity scores between the points of object and (in analogy to linear terms when using pairwise matching matrices). denotes the (non-negative) adjacency matrix of object (e.g. a matrix that encodes the Gaussian of pairwise Euclidean/geodesic distances between pairs of points). The matrix is a row/column reordering of the matrix according to the universe points, and the inner product between two reordered adjacency matrices computes their correlation, which we aim to maximise. For fixed , each term can be understood as the object-to-universe formulation of second-order matching terms when using pairwise matching matrices (analogous to the QAP in Koopmans-Beckmann form [25]). In compact matrix notation, Problem (10) can be written as

(11)

where , , and is the block-diagonal multi-adjacency matrix defined as . While (11) is based on the -matrix and hence intrinsically guarantees cycle-consistent multi-matchings, the objective function is a fourth-order polynomial that is to be maximised over the (binary) set .

3.2 Algorithm

In order to solve Problem (11) we propose to use an alternating higher-order projected power iteration, as outlined in Algorithm 1.

Input: similarities , multi-adjacency matrix
Output: cycle-consistent multi-matching
Initialise:
1for  do
       // step for quartic term
2      
       // step for quadratic term
3      
       // projection
4      
5       if   then
6            
7             return
Algorithm 1 Higher-order Projected Power Iteration (HiPPI) algorithm.

The main idea is to first perform a power iteration step with respect to the quartic term, followed by a power iteration step for the quadratic term, and eventually project onto the set . The approach is extremely simple, and merely comprises of matrix multiplications and the Euclidean projection onto . Similar as in other multi-matching approaches (e.g. [55, 45]), we have found that an initialisation based on solutions of linear matching problems is sufficient (cf. Sec. 4). Moreover, in all conducted experiments we have observed that Algorithm 1 is always able to improve the initial objective in (11), and that it typically terminates after iterations, see Fig. 1.

Figure 1: Left: the histogram shows for a given iteration count (horizontal axis) in what percentage of instances the algorithm was still running. Right: improvement of relative to the initial for ten random instances from Sec. 4 (log-scale).

For the sake of completeness, we state the following result:

Lemma 3.

Algorithm 1 terminates after a finite number of iterations.

Proof.

Since is a finite set, is bounded above for any . Moreover, since for any we have that (feasibility), the sequence produced from lines 11 of Alg. 1 is bounded and non-decreasing (by construction), and hence convergent. Since the are discrete, convergence implies that there exists a such that for all it holds that . Hence, Algorithm 1 terminates. ∎

Projection onto :

The projection onto the binary set can be solved as linear assignment problem:

Lemma 4.

The projection of onto the set , , is given by , where

(12)
Proof.

We have that . Moreover, means that is a binary matrix that has exactly a single element in each row that is . Hence, , and the only non-constant term that remains when minimising over is . When solving , the blocks in are decoupled, so that we can solve for them individually, i.e. for , we have

(13)
(14)
(15)

Lemma 4 shows that the projection onto can be performed by solving individual (partial) linear assignment problems, which can be performed efficiently with the Auction algorithm [7] (linear assignment problems with matrices of sizes in the order of can easily be solved).

Complexity analysis:

The update rule in Algorithm 1 can be written as . The matrix multiplications (lines 1 and 1 in Algorithm 1) have time complexity , where the product , with complexity must be computed only once and thus does not affect the per-iteration complexity. Since the projection amounts to solving independent (partial) linear assignment problems (each with sub-cubic empirical average time complexity [38]), the overall (average) per-iteration complexity is . The memory complexity is due to the matrix , which can be improved by considering sparse similarity scores.

Figure 2: Multi-image matching on the HiPPI dataset. Our method is the only one that considers geometric relations between points, while at the same time being scalable to such large datasets. We clearly outperform other matching methods as well as synchronisation methods in terms of the fscore (higher is better), while guaranteeing cycle-consistency, and being much faster compared to most competitors.

4 Experimental Results

In this section we extensively compare our method to other approaches on three datasets. To be more specific, we consider two multi-image matching datasets, Willow [12] and HiPPI, as well as the multi-shape matching dataset Tosca [8]. The datasets are summarised in Table 1.

Dataset Type Bij. #
HiPPI images no
Willow images yes
Tosca shapes no
Table 1: Overview of datasets. “Bij.” indicates whether the matchings are bijective, and “#” is the number of instances per dataset.

Similarity scores:

The similarity scores between image/shape and is encoded in the matrix , which is defined using feature matrices and of the respective image/shape, where is the feature dimensionality. As in [43], the similarity scores are based on a weighted Gaussian kernel, i.e.

(16)

where is a weight that depends on the distance between the features and the closest descriptor from the same image. For details we refer the reader to [43]. The particular choice of features for each dataset are described below.

Adjacency matrices:

The adjacency matrix of image/shape is based on Euclidean distances between pairs of 2D image point locations in the case of multi-image matching (or based on geodesic distances between pairs of points on the 3D shape surface in the case of multi-shape matching). By denoting the distance between the points with indices and as , the elements of the adjacency matrix are based on a Gaussian kernel, so that . We set , where for , and is a scaling factor.

Instance Pairwise Spectral MatchALS QuickMatch Wang et al. Ours
Car
Duck
Face
Motorbike
Winebottle
Table 2: Fscores (higher is better) and runtimes for the Willow dataset. All methods are initialised based on pairwise matches with linear costs (except QuickMatch [43], which is initialisation-free). Ours and Wang et al. [45] consider geometric relations between points.

4.1 HiPPI Dataset

In this experiment we compare various multi-image matching methods.

Dataset:

The HiPPI dataset comprises multi-image matching problems. For each problem instance, a (short) video sequence has been recorded (with resolution , frame-rate  FPS, and duration s). In each video, feature points and feature descriptors have been extracted using SURF [4] with three octaves. To obtain ground truth matchings, these feature points were automatically tracked across the sequence based on their geometric distance and feature descriptor similarity. To ensure reliable ground truth matchings, we have conducted the following three steps: (i) obvious wrong matchings between consecutive images have been automatically pruned, (ii) we have manually removed those features that were incorrectly tracked from the first to the last frame (by inspecting the first and last frame), and (iii) in order to prevent feature sliding in-between the first and last frame, we have manually inspected the flow of each remaining feature point and we have removed wrongly tracked points. Note that steps (ii) and (iii) have been performed by two different persons, which took in total about

hours. The multi-matching problems were then created by extracting evenly spaced frames from the sequence, where in each frame we added a significant amount of outlier points (randomly selected from the previously pruned points, where the number of points is chosen such that in each frame

of the points are outliers), and we simulate occlusions (a rectangle of size of the image dimensions) in order to get difficult partial multi-matching problems.

Multi-image matching:

We compare our method with QuickMatch [43], MatchEig [29], Spectral [34] (implemented by the authors of [55] for partial permutation synchronisation), and MatchALS [55]. The universe size is set to twice the average of the number of points per image, as in [29]. We have used QuickMatch () for initialising in our method, and we set . The results are shown in Fig. 2, where it can be seen that our method achieves a superior matching quality. Moreover, in contrast to other methods (except QuickMatch), our method guarantees cycle-consistency, while being significantly faster than MatchEig, Spectral and MatchALS.

#matches [13] Ours
cat
centaur
david
dog
horse
michael
victoria
Figure 3: Our method obtains significantly more multi-matchings (bottom right, see also Fig. 4) compared to the method of Cosmo et al. [13], while at the same time achieving comparable errors (percentage of correct keypoints, PCK).

4.2 Willow Dataset

For the evaluation on the Willow dataset [12], we use the experimental protocol from [45]

, where deep features have been used for matching (due to the large variation of the object appearances). For this dataset the matchings are bijective, and hence for all methods we set the universe size

to the number of annotated features. Since QuickMatch [43] is tailored towards partial matchings, as it implicitly determines the universe size during its internal clustering, we have found that it does not perform very well on this dataset (see Table 2). Hence, we initialise our method based on a spectral method applied to linear matching scores and we use . In Table 2 it can be seen that our method is superior compared to the other approaches.

4.3 Tosca Dataset

Figure 4: Qualitative results on the Tosca dataset for cat and centaur. Dots with same colour indicate matched points. QuickMatch (top) does not take geometric consistency into account and thus leads to mismatches (red circles). While Cosmo et al. [13] (centre) obtain only few reliable matchings, leading to regions without correspondences, our method (bottom) obtains significantly more reliable matchings.

Based on the experimental setup of [13] using the Tosca dataset [8], we compare our method with two other approaches that guarantee cycle-consistency, namely QuickMatch, and the sparse multi-shape matching approach by Cosmo et al. [13]. The feature descriptors on the shape surfaces are based on wave kernel signatures (WKS) [3], we use QuickMatch) as initialisation, and set .

Multi-shape matching:

Quantitative results are shown in Fig. 3 and qualitative results are shown in Fig. 4. As explained before, QuickMatch ignores geometric relations between points, and thus leads to geometric inconsistencies, as shown in the first row of Fig. 4. While the method of Cosmo et al. [13] is able to incorporate geometric relations between points, one major limitation of their approach is that only a sparse subset of matchings is found. This may happen even when the shape collection is outlier free [13]. This behaviour can be seen in the second row of Fig. 4, where only very few multi-matchings are obtained and hence there are large regions for which no correspondences are found. In contrast, our approach incorporates geometric consistency, produces significantly more multi-matchings (Fig. 4), and results in a percentage of correct keypoints (PCK) that is competitive to the method of Cosmo et al. [13], see Fig. 3.

5 Discussion & Limitations

Our method scales to much larger problems than previous methods that consider geometric information (cf. Figs. LABEL:fig:teaser2, and Table 2). We evaluated problems with up to about (, ), resulting in a runtime of about h. This is faster than the spectral approaches [34, 29] which take more than h while ignoring geometric relations. Other methods that consider geometric information [50, 40, 47, 45] are not applicable to such large problems, cf. Fig. LABEL:fig:teaser.

We have found that in all experiments the proposed update rule improves upon the initial obtained by a linear multi-matching, as illustrated in Fig. 1 right. This contrasts gradient-based approaches that cannot make steps large enough to improve upon the initial ,

as we show in the inset figure for projected gradient descent and the Frank-Wolfe method (FW) [19], where the latter optimises over the convex hull of .

Our formulation of the geometric consistency term in Problem (11) corresponds to the Koopmans-Beckmann form of the QAP [25], which is strictly less general compared to Lawler’s form  [26]. An interesting direction for future work is to devise an analogous algorithm for solving Lawler’s form (e.g. by rewriting as , cf. [54], and performing consecutive power iteration steps).

6 Conclusion

We presented a higher-order projected power iteration approach for multi-matching. Contrary to existing permutation synchronisation methods [34, 11, 55, 39, 2, 29], our method is able to take geometric relations between points into account. Hence, our approach can be seen as a generalisation of permutation synchronisation. Moreover, previous multi-matching methods that consider geometric consistency (e.g. [50, 47]) only allow to solve problems with up to few thousand points. In contrast, we demonstrated that our approach scales to tens of thousands of points.

In addition to being able to account for geometric consistency, key properties of our method are computational efficiency, simplicity, and guaranteed cycle-consistency. Moreover, we have demonstrated superior performance on various datasets, which highlights the practical relevance of the proposed algorithm.

Acknowledgements

This work was funded by the ERC Consolidator Grant 4DRepLy. We thank Franziska Müller for providing feedback on the manuscript.

References

  • [1] Y. Aflalo, A. Bronstein, and R. Kimmel. On convex relaxation of graph isomorphism. Proceedings of the National Academy of Sciences, 112(10):2942–2947, 2015.
  • [2] F. Arrigoni, E. Maset, and A. Fusiello. Synchronization in the Symmetric Inverse Semigroup. In ICIAP, 2017.
  • [3] M. Aubry, U. Schlickewei, and D. Cremers. The wave kernel signature: A quantum mechanical approach to shape analysis. In ICCV Workshops, 2011.
  • [4] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3):346–359, 2008.
  • [5] M. S. Bazaraa and A. N. Elshafei. An exact branch-and-bound procedure for the quadratic-assignment problem. Naval Research Logistics Quarterly, 26(1):109–121, 1979.
  • [6] F. Bernard, C. Theobalt, and M. Moeller. DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems. In CVPR, 2018.
  • [7] D. P. Bertsekas. Network Optimization: Continuous and Discrete Models. Athena Scientific, 1998.
  • [8] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Numerical geometry of non-rigid shapes. Springer Science & Business Media, 2008.
  • [9] R. Burkard, M. Dell’Amico, and S. Martello. Assignment problems. 2009.
  • [10] Y. Chen and E. Candes. The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences. Communications on Pure and Applied Mathematics, 71(8), 2018.
  • [11] Y. Chen, L. J. Guibas, and Q.-X. Huang. Near-Optimal Joint Object Matching via Convex Relaxation. In ICML, 2014.
  • [12] M. Cho, K. Alahari, and J. Ponce. Learning graphs to match. In ICCV, 2013.
  • [13] L. Cosmo, E. Rodolà, A. Albarelli, F. Mémoli, and D. Cremers. Consistent partial matching of shape collections via sparse modeling. In Computer Graphics Forum, volume 36, pages 209–221, 2017.
  • [14] T. Cour, P. Srinivasan, and J. Shi. Balanced graph matching. NIPS, 2006.
  • [15] L. De Lathauwer, P. Comon, B. De Moor, and J. Vandewalle. Higher-order power method. Nonlinear Theory and its Applications, NOLTA95, 1, 1995.
  • [16] O. Duchenne, F. Bach, I.-S. Kweon, and J. Ponce. A tensor-based algorithm for high-order graph matching. IEEE transactions on pattern analysis and machine intelligence, 33(12):2383–2395, 2011.
  • [17] N. Dym, H. Maron, and Y. Lipman. DS++ - A flexible, scalable and provably tight relaxation for matching problems. ACM Transactions on Graphics (TOG), 36(6), 2017.
  • [18] F. Fogel, R. Jenatton, F. Bach, and A. d’Aspremont. Convex Relaxations for Permutation Problems. In NIPS, 2013.
  • [19] M. Frank and P. Wolfe. An algorithm for quadratic programming. Naval Research Logistics (NRL), 3(1-2):95–110, Mar. 1956.
  • [20] G. H. Golub and C. F. V. Loan. Matrix Computations. The Johns Hopkins University Press, 1996.
  • [21] T. Heimann and H.-P. Meinzer. Statistical shape models for 3D medical image segmentation: A review. Medical Image Analysis, 13(4):543–563, 2009.
  • [22] Q.-X. Huang and L. Guibas. Consistent shape maps via semidefinite programming. In Symposium on Geometry Processing, 2013.
  • [23] B. Jiang, J. Tang, C. Ding, and B. Luo. Binary Constraint Preserving Graph Matching. In CVPR, 2017.
  • [24] I. Kezurer, S. Z. Kovalsky, R. Basri, and Y. Lipman. Tight Relaxation of Quadratic Matching. Comput. Graph. Forum, 2015.
  • [25] T. C. Koopmans and M. Beckmann. Assignment Problems and the Location of Economic Activities. Econometrica, 25(1):53, Jan. 1957.
  • [26] E. L. Lawler. The quadratic assignment problem. Management science, 9(4):586–599, 1963.
  • [27] M. Leordeanu and M. Hebert. A Spectral Technique for Correspondence Problems Using Pairwise Constraints. In ICCV, 2005.
  • [28] E. M. Loiola, N. M. M. de Abreu, P. O. B. Netto, P. Hahn, and T. M. Querido. A survey for the quadratic assignment problem. European Journal of Operational Research, 176(2):657–690, 2007.
  • [29] E. Maset, F. Arrigoni, and A. Fusiello. Practical and Efficient Multi-View Matching. In ICCV, 2017.
  • [30] J. Munkres. Algorithms for the Assignment and Transportation Problems. Journal of the Society for Industrial and Applied Mathematics, 5(1):32–38, Mar. 1957.
  • [31] A. Nguyen, M. Ben-Chen, K. Welnicka, Y. Ye, and L. J. Guibas. An Optimization Approach to Improving Collections of Shape Maps. Computer Graphics Forum, 30(5):1481–1491, 2011.
  • [32] Q. Nguyen, A. Gautier, and M. Hein. A flexible tensor block coordinate ascent scheme for hypergraph matching. 2015.
  • [33] C. Olsson, A. P. Eriksson, and F. Kahl. Solving Large Scale Binary Quadratic Problems - Spectral Methods vs. Semidefinite Programming. CVPR, 2007.
  • [34] D. Pachauri, R. Kondor, and V. Singh. Solving the multi-way matching problem by permutation synchronization. In NIPS, 2013.
  • [35] P. M. Pardalos, F. Rendl, and H. Wolkowicz. The Quadratic Assignment Problem - A Survey and Recent Developments. DIMACS Series in Discrete Mathematics, 1993.
  • [36] H.-M. Park and K.-J. Yoon. Consistent multiple graph matching with multi-layer random walks synchronization. Pattern Recognition Letters, 2018.
  • [37] C. Schellewald and C. Schnörr. Probabilistic subgraph matching based on convex relaxation. In EMMCVPR, 2005.
  • [38] B. L. Schwartz. A computational analysis of the Auction algorithm. European Journal of Operational Research, 74(1):161–169, 1994.
  • [39] Y. Shen, Q. Huang, N. Srebro, and S. Sanghavi. Normalized Spectral Map Synchronization. In NIPS, 2016.
  • [40] X. Shi, H. Ling, W. Hu, and J. Xing. Tensor power iteration for multi-graph matching. In CVPR, 2016.
  • [41] P. Swoboda, C. Rother, H. A. Alhaija, D. Kainmüller, and B. Savchynskyy. Study of lagrangean decomposition and dual ascent solvers for graph matching. In CVPR, 2017.
  • [42] L. Torresani and V. Kolmogorov. A dual decomposition approach to feature correspondence. TPAMI, 35(2):259–271, 2013.
  • [43] R. Tron, X. Zhou, C. Esteves, and K. Daniilidis. Fast Multi-Image Matching via Density-Based Clustering. In ICCV, 2017.
  • [44] M. Vestner, R. Litman, E. Rodolà, A. M. Bronstein, and D. Cremers. Product Manifold Filter - Non-Rigid Shape Correspondence via Kernel Density Estimation in the Product Space. CVPR, 2017.
  • [45] Q. Wang, X. Zhou, and K. Daniilidis. Multi-Image Semantic Matching by Mining Consistent Features. In CVPR, 2018.
  • [46] M. L. Williams, R. C. Wilson, and E. R. Hancock.

    Multiple graph matching with Bayesian inference.

    Pattern Recognition Letters, 1997.
  • [47] J. Yan, M. Cho, H. Zha, and X. Yang. Multi-graph matching via affinity optimization with graduated consistency regularization. TPAMI, 38(6):1228–1242, 2016.
  • [48] J. Yan, Y. Li, W. Liu, H. Zha, X. Yang, and S. M. Chu. Graduated Consistency-Regularized Optimization for Multi-graph Matching. ECCV, 2014.
  • [49] J. Yan, Y. Tian, H. Zha, X. Yang, Y. Zhang, and S. M. Chu. Joint Optimization for Consistent Multiple Graph Matching. ICCV, 2013.
  • [50] J. Yan, J. Wang, H. Zha, and X. Yang. Consistency-driven alternating optimization for multigraph matching: A unified approach. IEEE Transactions on Image Processing, 2015.
  • [51] J. Yan, H. Xu, H. Zha, X. Yang, and H. Liu. A matrix decomposition perspective to multiple graph matching. In ICCV, 2015.
  • [52] M. Zaslavskiy, F. Bach, and J.-P. Vert. A Path Following Algorithm for the Graph Matching Problem. TPAMI, 31(12):2227–2242, 2009.
  • [53] Q. Zhao, S. E. Karisch, F. Rendl, and H. Wolkowicz. Semidefinite programming relaxations for the quadratic assignment problem.

    Journal of Combinatorial Optimization

    , 2(1):71–109, 1998.
  • [54] F. Zhou and F. De la Torre. Factorized Graph Matching. TPAMI, 38(9):1774–1789, 2016.
  • [55] X. Zhou, M. Zhu, and K. Daniilidis. Multi-image matching via fast alternating minimization. In ICCV, 2015.