Range scan registration has attracted broad interests due to its wide applications in robot mapping Henry2012RGB; Borrmann2008Globally; N20076D, 3D model reconstruction Zhou2013Dense; Izadi2011KinectFusion, object recognition Abate07; Held16
and etc. The task of registration is to calculate the optimal transformation for two or more range scans so as to transfer them into one coordinate system and recover the original scene of a 3D object. Based on the number of scans to be registered, this problem can be classified into two categories: pair-wise registration and multi-view registration. The most popular method for pair-wise registration is the iterative closest point (ICP) algorithm proposed by Besl et al.Besl92, which iteratively builds up correspondences and calculates the optimal transformation by minimizing the residual error. Although the ICP algorithm has good performance in efficiency, it is a local convergent approach. Besides, this approach can not be applied to the registration of scan pair, which contains large non-overlapping areas. Therefore, a lot of ICP variants were proposed for the pair-wise registration.
To address non-overlapping areas, Chetverikov et al. Chetverikova05 proposed the trimmed ICP (TrICP) algorithm, which introduced an overlap percentage into the original ICP algorithm. During each iteration, it requires to search an optimal overlap percentage, so it is time-consuming. Subsequently, Phillips et al. Phillips07 proposed an efficient ICP variant called the fractional TrICP (FTrICP) algorithm, which can simultaneously compute the overlap percentage and rigid transformation for partially overlapping scans. For the local convergence issue, Fitzgibbon et al. Fitzgibbon03 employed the Levenberg-Marquardt algorithm to expand the narrow convergent range of ICP algorithm. Besides, invariant features were introduced into ICP algorithm by Lee et al. Sharp2002ICP
. Moreover, the genetic algorithmLomonosov2006Pre; Zhu2014Robust and particle filter Sandhu2010Point were utilized to search the optimal rigid transformation. To boost the accuracy, some probabilistic methods Granger02; Jian11; Myronenko10; Tsin04 were also proposed. Much precise these methods may be, the huge computational resources they require poses a great challenge to most application areas. To improve the robustness, some methods have also been investigated in recent years. Based on the ratio of bidirectional distances, Zhu et al. Zhu2016Registration
proposed an ICP variant, which assigns a probability for each correspondence. Besides, Xu et al.Hasanbelliu2014Information introduced the concept of correntropy into pair-wise registration and proposed the approach to achieve the pair registration by maximizing the correntropy of one scan pair.
Although these approaches may obtain good results for pair-wise registration, they are not suitable for the multi-view registration. Therefore, many researchers explore the principle of pair-wise registration and extend it to solve the multi-view registration. The original approach was proposed by Chen et al. chen1992object, which repeatedly aligns two scans and merge them into one model until all range scans are integrated into the whole model. This approach is simple and efficient, but it suffers from the problem of error accumulation. To address this issue, Bergevin et al. bergevin1996towards proposed an ICP based registration approach, which can simultaneously align one scan to all the other scans. Since this approach should establish point correspondences between one scan and all the others, it is very time-consuming. Subsequently, the multi-z-buffer technique benjemaa1999fast was then introduced into multi-view registration to improve the efficiency. To further reduce the accumulative error, some other approaches shih2008efficient; torsello2011multiview view multi-view registration as the optimization problem over the graph of adjacent scans, which transfer the registration error between coordinate systems. As these approaches do not need to update the point correspondences during registration, they cannot really reduce accumulative error, but just distribute it over all scans.
Recently, Mateo et al. mateo2014bayesian
utilized the Bayesian framework to deal with missing data of pair-wise correspondences in multi-view registration, which then be solved by the Expectation-Maximization algorithm. Related works can also be found inevangelidis2014generative. Besides, Toldo et al. toldo2010global proposed an ICP and Generalized Procrustes Analysis beinat2001generalised combined approach to achieve multi-view registration. To explore the redundant information of non-adjacent range scans, Godvin et al. govindu2014averaging introduced the Lie-Algebraic averaging govindu2004lie algorithm to refine global motions. This approach was then extended by Li et al. Zhongyu2014Improved to achieve more accurate and efficient multi-view registration. Besides, Guo et al. Guo2017Weighted proposed a weighted motion averaging algorithm to improve the accuracy of multi-view registration. More recently, Arrigoni et al. arrigoni2016global cast the multi-view problem into the framework of the low-rank and sparse (LRS) matrix decomposition. By decomposing the relative motion stacked matrix, the noise matrix is discarded and low rank matrix can be obtained to recover global motions for multi-view registration. However, this approach is sensitive to the sparsity of the stacked matrix to be decomposed. Besides, it treats all relative motions equally in spite of their varied reliabilities, which is not good for multi-view registration.
In this paper, we extend the approach presented in arrigoni2016global to achieve more effective registration of multi-view range scans. The contribution of this paper can be delivered as follows: a matrix completion strategy is proposed to reduce the sparsity of potentially decomposed matrix based on the anti-symmetry property of relative motions (block elements). Then, a weight value is estimated and assigned to denote the reliability of each non-zero block elements. Moreover, the L1-ALM zheng2012practical algorithm is extended to decompose the weighted matrix and obtain accurate global motions for multi-view registration. To demonstrate its effectiveness, the proposed approach was also tested on public available data sets.
The rest of this paper is organized as follows: Section 2 briefly introduces the LRS decomposition framework for the multi-view registration. Then in Section 3, the proposed approach is presented in details. Following that is Section 4, in which the proposed approach was tested and compared with some related approaches. Finally, some conclusions and future works are drawn in Section 5.
2 LRS decomposition for multi-view registration
Suppose there are range scans, which are acquired from one object in different views. Let be the global motion, which denotes the rigid transformation between the local reference frame of the th range scan and the global coordinate system:
represent the rotation matrix and the translation vector, respectively. Obviously, the rank ofis 4. Given initial global motions, the task of multi-view registration is to estimate accurate global motions for all range scans. Without loss of generality, the global coordinate system can be attached to the local reference frame of the first range scan. Related to global motion, there is another kind of motion called as the relative motion , which represents the rigid transformation between the reference frame of the th scan and that of the th scan:
where denotes the inverse of motion and the rank of is also 4.
For the multi-view registration, Arrigoni et al. arrigoni2016global proposed the LRS matrix decomposition based approach. Before presenting this approach, three block matrix should be introduced and defined as:
where indicates the identity matrix. Therefore, and . Although the matrix is larger than or , they have the same rank due to the special structure of .
For the multi-view registration, the LRS decomposition based approach views each available relative motions as a block element to reconstruct the matrix
, where the non-available relative motions are replaced by zero matrix. As the reconstructed matrixis the approximation of , they have the following relation:
is called as error matrix, which is a sparse matrix containing noises and outliers.
According to arrigoni2016global, the multi-view registration appcan be formulated as the following optimization problem:
where represents the projection of onto and is an indicator matrix indicating whether the corresponding block element in the reconstructed matrix is available or not. Subsequently, the LRS decomposition algorithm can be applied to solve Eq. (6) and obtain the matrix , which is used to recover global motions for muti-view registration.
Although the framework of LRS decomposition for multi-view registration has been proposed in arrigoni2016global, this approach is sensitive to the sparsity of reconstructed matrix. Besides, it treats all block elements equally in spite of their varied reliability. To obtain good registration results, more effective approach is required.
3 The proposed approach
Although the LRS decomposition has been introduced to solve the multi-view registration arrigoni2016global, two limitations should be stated. Firstly, this approach is sensitive to the sparsity of reconstructed matrix to be decomposed. Then, during matrix decomposition, it treats all blocking elements equally and does not take varied reliability of each relative motion into consideration, which may lead to unexpected registration results. To address these two issues, we propose an effective LRS decomposition based approach for multi-view registration and its flowchart is displayed in Fig. 1. As shown in Fig. 1, the proposed approach consists of the following four major steps, which will be presented with more details.
3.1 Matrix reconstruction
For the matrix reconstruction, it is required to obtain relative motions, which can be estimated by the pair-wise registration approach. By consideration both the efficiency and accuracy, the trimmed ICP algorithm is utilized to estimate the relative motion of scan pair.
Suppose there are two partially overlapping range scans in , a data shape and a model shape . Given initial rigid transformation, the TrICP algorithm achieves the pair-wise registration by minimizing the following objective function:
where denotes the overlap percentage parameter, represents the overlapping part of data shape to model shape , represents the cardinality of set, is the correspondence of and (=2) is a preset parameter.
3.1.1 Estimation of overlapping percentage
Although the TrICP algorithm is effective for pair-wise registration, it only suitable for the registration of scan pair with a certain amount of overlap percentages. To obtain reliable relative motions, the TrICP algorithm can only applied to these scan pairs that satisfy , where is a predefined threshold. Therefore, it is required to estimate the overlap percentage for each scan pair before pair-wise registration. To address this issue, the method proposed in Zhongyu2014Improved is directly utilized. For each point in the th range scan, it firstly searches correspondences from each other scans. According to their distances, these point pairs can be sorted in ascending order. By traversing each sorted point pair, all its front point pairs can be used to calculate the value of objective function (7). The distance of the point pair, which minimizes the objective function, can be viewed as the distance threshold. For the th range scan, if there are point pairs, whose distances are smaller than the distance threshold, then the overlap percentage is estimated as follows:
where denotes the number of point in the th range scan. It should be noted that and are two different overlap percentages, which are always unequal. For each scan pair, if its overlap percentage satisfies , then the th range scan and the th range scan can be viewed as the data shape and model shape, respectively. Further, the TrICP algorithm can be directly used to estimate its relative motion . Here, can guarantee the TrICP algorithm to achieve reliable pair-wise registration.
3.1.2 Estimation of weight
Actually, the reliability of each relative motion are varied due to many reasons, such as noise level, resolution of range point and overlap percentage of scan pair. To indicate its reliability, a weight requires to be estimated for each relative motion. Intuitively, the smaller the trimmed mean square error (MSE) is, the more reliable the relative motion is. Before introducing the estimation method, some factors related to the trimmed MSE should be presented and analyzed. As shown in Fig. 2, accurate pair-wise registration will lead to small trimmed MSE. However, the trimmed MSE also related to the point resolution of model shape. More specifically, with the same registration accuracy, high resolution of model shape will lead to small trimmed MSE. Vice vera. Accordingly, a weight for the relative motion can be reasonably estimated as follows:
where indicates the point resolution of model shape and denotes the trimmed MSE of aligned scan pair, which can be directly obtained by the TrICP algorithm. More specifically, they can be calculated as follows:
where denotes the distance of one point pair located in the overlapping areas, represents the distance of one point in the model shape to its nearest neighbor in the model shape itself. According to the definition of weight, reliable relative motion will lead to large weight.
As there maybe large difference between varied relative motions, it is better to normalize all weights as follows:
where denotes the maximum value in . It should also be noticed that this is the weight for each relative motion. To associate with each element of , the following expanding is required:
where indicates the Kronecker product and denotes a matrix filled by ones.
After the pair-wise registration, the relative motion can be utilized to reconstruct the , which is viewed as the approximation of the low rank matrix . In the same way, its corresponding weight can be utilized to reconstruct the matrix .
3.2 Matrix Completion
As the LRS decomposition algorithm is sensitive to the sparsity of reconstructed matrix to be decomposed, it is better to obtain as many relative motions as possible so as to reconstruct a matrix with reduced sparsity. Therefore, it is required to design a completion strategy for the matrix reconstruction.
According to Eq. (2), the relative motion can be calculated as:
Obviously, there exits the property of anti-symmetry between the pair of relative motions , which is formulated as:
where indicates the inverse of motion . For a pair of scans acquired from one object, two different overlap percentages are estimated by the proposed method. Suppose one of them is larger than the predefined threshold and the other is smaller than . Subsequently, one relative motions with a weight can be available to reconstruct the . To reduce the sparsity of , the anti-symmetry property of relative motion can be applied to obtain the other relative motion with its weight assigned as:
As shown in Fig. 1, this matrix completion strategy can seriously reduce the sparsity of reconstructed matrix to be decomposed and will lead to robust results of LRS matrix decomposition.
3.3 Weighted LRS decomposition
After matrix completion, the matrix has been reconstructed with less missing data. Moreover, the weight matrix is also provided to indicate the reliability of component at the same position in . Therefore, the weighted LRS decomposition should be designed to approximate the low rank matrix .
According to zheng2012practical, it is reasonable to use the robust 1-norm as the measurement to approximate . The approximation can be formulated as the optimization problem:
where is a weight matrix, which indicates the reliability of component at the same position in . As mentioned before, and . Therefore, . By replacing the trace-norm regularizer, Eq. (16) can be reformulated as:
After the investigation of this problem, we find that the Augmented Lagrange Multiplier (ALM) method can be utilized to solve it.
Benefited from the ALM algorithm, the corresponding augmented Lagrange function can be derived as:
where is the Lagrange multiplier, denotes the penalty parameter and indicates the inner product of two matrices. As it is somewhat same to the problem discussed in zheng2012practical, this problem can also be solved the Gauss-Seidel Iteration algorithm. In each iteration, three steps are alternately utilized to estimate , and , respectively.
3.3.1 Solving via Orthogonal Procrustes
By fixing and , the update of can be simplified as the following problem:
The above orthogonal procrustes problem can be solved by the SVD method of :
Consequently, can be derived as:
3.3.2 Solving via Singular Value Decomposition
Given and , the update of can be achieved by solving:
Since , the above objective function can be reformulated as:
To solve problem (23), the soft-thresholding (shrinkage) operator is adopted:
is the sign function. Then the singular values ofis computed as:
Finally, the shrinkage operator is applied to the singular values and the optimal can be updated as:
3.3.3 Solving via Absolute Value Shrinkage
Provided with and , the error matrix can be updated as follows:
Since some block element of is unobservable, the update of should be divided into two parts. Corresponding to the observable part of , the elements of can be updated by the absolute value shrinkage:
where and denotes the ceiling operation of each element of . While, elements corresponding to missing entries of should be updated by:
where denotes the complement of and .
The weighted LRS decomposition algorithm is summarized in Algorithm 1. By the application of this algorithm, two matrix and are obtained to approximate .
3.4 Recovery of global motions
After the LRS decomposition, we obtain . Theoretically, these block elements located in the first column of can directly be viewed as global motions for multi-view registration. However, these block elements may not be the elements of Special Euclidean groups due to no constraint imposed on matrix decomposition. Accordingly, some operations are required to recover each global motion from one block elements located in the first column of . Firstly, the corresponding block element is assign to , then is normalized by its element as . Besides, three elements
should be assigned with the zero value. Finally, the Singular Value Decomposition (SVD) can be applied toas follows:
and represents a diagonal matrix with the elements of on the main diagonal. After these operations, each block element located in the first column of can be recovered as one global motion for muti-veiw registration.
3.5 Implementation details
Accordingly, the overall process of the proposed multi-view registration approach is summarized in Algorithm 2.
With the matrix completion, the proposed approach can improve the robustness and efficiency of the LRS decomposition for multi-view registration. By proposing the weighted LRS decomposition, the accuracy of multi-view registration can be increased.
To show the performance of the proposed approach, experiments were conducted on seven data sets from the Stanford 3D Scanning Repository Stanford and UWA 3D Modeling Dataset UWA. Registration results are reported in the form of the objective function value (), which was designed in Zhu2014Surface. As all multi-view registration approaches can estimate the optimal global motion , it is convenient to define the operation and reconstruct the integrated model:
Then, one special model is defined for each scan as follows:
For the accuracy evaluation, the is calculated as follows:
where denotes the function displayed in Eq. (7). To establish point correspondences, the tree based method was adopted to search the nearest-neighbor. All codes were implemented in Matlab on a desktop with four-core 3.6GHz processor and 8GB of memory.
To validate the proposed approach, it are compared with three versions of LRS decomposition based multi-view registration approaches: original LRS decomposition (LRS), LRS with matrix completion (LRS with MC), weighted LRS decomposition (Weighted LRS). For each data set, the same initial parameters were provided for four approaches. Registration results are shown in the form of and run time. Tab 1 displays registration results for all LRS decomposition based approaches. As shown in Tab 1, the original LRS decomposition may get the worst results for multi-view registration. Compared with the original LRS decomposition, the introduction of matrix completion and motion weight can both improve the performance of multi-view registration. Moreover, the integration of matrix completion and motion weight leads to development of the proposed approach, which can always obtain the best results for multi-view registration.
As stated before, the matrix for LRS decomposition is reconstructed by available relative motions, which are estimated by the pair-wise registration approach. However, most pair-wise registration approaches are unable to obtain reliable results for scan pairs with low overlap percentage, so the reconstructed matrix are always sparse. Since the LRS decomposition is sensitive to the sparsity of the reconstructed matrix, it is difficult to obtain good results. To reduce the sparsity, some block elements of the reconstructed matrix can be completed due to the anti-symmetry property of relative motions. With the reduced sparsity, the robustness of LRS decomposition is increased, which can lead to good multi-view registration. As relative motions of scan pairs are estimated by the pair-wise registration, their reliability are varied due to different overlap percentages of scan pairs. In the original LRS decomposition, the varied reliability is ignored, which is harmful for multi-view registration. By the analysis of pair-wise registration, we use the trimmed MSE and resolution of model shape to calculate the weight, which denotes the reliability of each relative motion. By introducing the weight in LRS decomposition, the accuracy of multi-view registration is increased. What’s more, the combination of matrix completion and motion weight can further improve the performance of LRS decomposition for multi-view registration.
|LRS||LRS with MC||Weighted LRS||Ours|
|MAICP govindu2014averaging||CFTrICP Zhu2014Surface||LRS arrigoni2016global||Ours|
Contrary to the intuition, the proposed approach is the most efficient among all variants of LRS decomposition based approaches. Although some time is required by the matrix completing and weight calculation, it is only a small part of time spent on the multi-view registration. Usually, the most time-consuming operation is the establishment of point correspondence, which is included in pair-wise registration of LRS decomposition based approaches. For one special scan pair, fine initial parameters will cost less time to achieve accurate pair-wise registration. As shown in Fig. 1, pair-wise is the basis of multi-view registration, which provides the initial parameters for pair-wise registration in return. Since both matrix completing and motion weight can lead to robust and accurate multi-view registration, they can provide good initial parameters for pair-wise registration. Therefore, both matrix completing and motion weight can accelerate pair-wise registration, so they improve the efficiency of multi-view registration.
To illustrate its performance, the proposed approach was compared with three state-of-the-art approaches, there are the motion averaging with the TrICP algorithm (MAICP) govindu2014averaging, the coarse to fine registration approach (CFTrICP) Zhu2014Surface, and the original LRS decomposition based approach (LRS) arrigoni2016global. Results of multi-view registration are also measured in the form of and run time.
4.2.1 Accuracy and efficiency
For the comparison of accuracy and efficiency, experiments were carried on seven data sets with the same initial parameters. Comparison results of all competed approaches are displayed in Tab. 2. To illustrate the comparison in a more intuitive manner, Fig. 3 displays the registration results of five data sets for all competed approaches in the form of cross-sections. As shown in Tab. 2 and Fig. 3, the proposed approach can always obtain good results of multi-view registration. While, other approaches can not always achieve good multi-view registration.
As the pair-wise registration is the basis of multi-view registration, MAICP utilizes the TrICP algorithm to estimate relative motions of some scan pairs with high overlap percentages and then views these relative motions as the input of motion averaging algorithm to compute global relative motions. In MAICP, one unreliable relative motions will lead to inaccurate results, even other relative motions are very reliable. Therefore, only fine initial parameters can lead to good multi-view registration. With other global motions fixed, CFTrICP alternately refines each global motion by the TrICP algorithm, so its final registration result is always better than initial results. But this approach is easy to trap into local minimum. For good registration, CFTrICP requires to be provided with fine initial parameters. Otherwise, it is difficulty to achieve good registration.
As MAICP, LRS decomposition based approach also utilizes a set of relative motions to recover global motions for the multi-view registration. It is robust to unreliable relative motions but sensitive to the sparsity of reconstructed matrix. Without the matrix completion, the reconstructed matrix are always sparse, which will lead to the failure of LRS decomposition. By introducing the matrix completion, the proposed approach can always obtain the robust LRS decomposition results for multi-view registration. What’s more, the weight of relative motions arrows the LRS decomposition to pay more attention to reliable relative motions, which can further improve the performance of LRS decomposition for the multi-view registration. Therefore, the proposed approach can almost obtain the best registration results among all competed approaches.
To compare the robustness, all competed approaches were tested on Stanford Dragon with different groups of initial parameters, which were acquired by adding some uniformly random noises to the rotation matrix. To eliminate the randomness, 20 Monte Carlo (MC) trials were carried out with respect to each noise level. For comparison, mean value of and run time are displayed in Tab. 3. As shown in Tab. 3, the proposed approach obtain the most accurate registration results under varied noise levels. Although all other approaches can obtain good registration results under low noise level, their performance will decrease seriously with the increase of noise level.
|MAICP govindu2014averaging||CFTrICP Zhu2014Surface||LRS arrigoni2016global||Ours|
Similar to LRS decomposition based approach, MAICP also recovers all global motions form a set of relative motions, which estimated by the TrICP algorithm. However, this approach is sensitive to unreliable relative motions and one unreliable relative motion will lead to the failure of multi-view registration. Under high noise level, it is difficulty to accurately estimate the overlap percentage of each scan pair, which will certainly introduce the unreliable pair-wise registration. Hence, the performance of MAICP turn to be seriously decreased. Different from MAICP, CFTrICP utilizes the TrICP algorithm to refine each global motion alternately, which make it easy to trap into local minimum. Under low noise level, initial global motions are accurate and they can be easily refined. However, with the increase of noise level, CFTrICP may be convergent to local minimum quickly due to inaccurate initial parameters and global motions are diffculty to be refinded.
Although LRS decomposition based approach is robust to a small portion of unreliable relative motions, it is sensitive to the sparsity of the reconstructed matrix. Under low noise level, a set of reliable relative motions are available to reconstruct the matrix for LRS matrix decomposition, which may result in good multi-view registration. With the increase of noise level, some available relative motions turn to be unreliable, which can reduce the sparsity of the reconstructed matrix and lead to the failure of multi-view registration. By introducing the matrix completion, the sparsity of the reconstructed matrix is reduced, which increase the robustness of LRS decomposition. Besides, the weight of relative motions allows the LRS decomposition pay more attention to these reliable relative motions. Therefore, the proposed approach can achieve multi-view registration with good performance even under high noise levels.
This paper proposes a novel approach for multi-view registration based on the weighted LRS matrix decomposition. According to the anti-symmetry property of relative motions, it firstly applies the completion strategy to reduce the sparsity of reconstructed matrix to be decomposed. As the LRS decomposition algorithm is sensitive to the sparsity of potentially decomposed matrix, the completion strategy can improve its robustness. Additionally, it introduces the weight to indicate the reliability of each block element of the reconstructed matrix and then proposes the weighted LRS decomposition algorithm. This algorithm can pay more attention to reliable block elements with large weight and achieve more accurate multi-view registration. Besides, compared with the original LRS decomposition, the proposed approach can also make the progress in efficiency for multi-view registration. Experiments on public available data sets demonstrate its good performance over the state-of-the-art approaches on robustness, accuracy, and efficiency.
Although the proposed approach has good performance for the multi-view registration, it does not mean that this approach can solve any multi-view registration problem. As shown in Fig. 4, multi-view range scans are transformed into a model graph, where each circle indicates one range scan and each line with arrow denotes one available relative motion. Actually, the proposed approach can only achieve the multi-view registration of these range scans, which can be transformed into a completed model. It is not suitable for the multi-view registration of these range scans, which can only be denoted by several partial models. However, it should be noted that many approaches for multi-view registration proposed so far share this limitation as well.
Similar to most of multi-view registration approaches, the proposed approach should be provided with initial global motions. Therefore, our future work will focus on the estimation of initial global motions for the multi-view registration.
This work is supported by the National Natural Science Foundation of China under Grant No. 61573273 and Natural Science Foundation of Jiangsu Province under Grant No. BK20161516. It is also supported by State Key Laboratory of Rail Transit Engineering Informatization (FSDI) under Grant No. SKLK16-09. Besides, we would like to thank Federica Arrigoni for providing the MATLAB implementation of arrigoni2016global.