IGSP
Iterative Global Similarity Points (IGSP) algorithm
view repo
In this paper, we propose a coarse-to-fine integration solution inspired by the classical ICP algorithm, to pairwise 3D point cloud registration with two improvements of hybrid metric spaces (eg, BSC feature and Euclidean geometry spaces) and globally optimal correspondences matching. First, we detect the keypoints of point clouds and use the Binary Shape Context (BSC) descriptor to encode their local features. Then, we formulate the correspondence matching task as an energy function, which models the global similarity of keypoints on the hybrid spaces of BSC feature and Euclidean geometry. Next, we estimate the globally optimal correspondences through optimizing the energy function by the Kuhn-Munkres algorithm and then calculate the transformation based on the correspondences. Finally,we iteratively refine the transformation between two point clouds by conducting optimal correspondences matching and transformation calculation in a mutually reinforcing manner, to achieve the coarse-to-fine registration under an unified framework.The proposed method is evaluated and compared to several state-of-the-art methods on selected challenging datasets with repetitive, symmetric and incomplete structures.Comprehensive experiments demonstrate that the proposed IGSP algorithm obtains good performance and outperforms the state-of-the-art methods in terms of both rotation and translation errors.
READ FULL TEXT VIEW PDF
Registration is a fundamental but critical task in point cloud processin...
read it
RGB-D scanning of indoor environments is important for many applications...
read it
Removing outlier correspondences is one of the critical steps for succes...
read it
3D local feature extraction and matching is the basis for solving many t...
read it
Intrinsic isometric shape matching has become the standard approach for ...
read it
In the last decades, some literature appeared using the Lie groups theor...
read it
A popular paradigm for 3D point cloud registration is by extracting 3D
k...
read it
Iterative Global Similarity Points (IGSP) algorithm
As the development of laser scanning and photogrammetry, point cloud, which depicts the world in 3D manner, has been widely collected by many platforms like airborne laser scanning (ALS) and terrestrial laser scanning (TLS). Point cloud registration is a fundamental problem in 3D computer vision and photogrammetry. Given several sets of points in different coordinate systems, the aim of registration is to find the transformation that best aligns all of them into a common coordinate system. Point cloud registration plays a significant role in many vision applications such as 3D model reconstruction
[8][40], cultural heritage management[28][42], landslide monitoring[38] and solar energy analysis[19].In 3D object recognition, fitness degree between an existing model object and an extracted object in the scene can be evaluated with registration results [6][21].In the field of robotics, for simultaneous localization and mapping(SLAM), the registration can act as a visual odometry to realize structure from motion and locate the current view into the global scene[4][34].Generally, there are four key challenges for unordered point clouds registration: (1) uneven point densities , (2) the huge amount of data , (3) repetitive, symmetric, and incomplete structures , and (4) limited overlaps between point clouds . All of these challenges can seriously affect the performance of point cloud registration methods. To address these challenges, extensive studies have been done to improve the accuracy, efficiency, and robustness of point cloud registration. Point cloud registration can be roughly categorized into pairwise and multi-view registration according to the number of input point clouds[20]. The pairwise registration is the prerequisite of multi-view registration, which is the focus of this paper. The registration process can be further divided into two major steps: coarse registration, in which an initial transformation between two point clouds is estimated, and fine registration, in which the initial transformation is then further refined[17].
The remainder of this paper is organized as follows. Following this introduction, Section 2 briefly reviews the representative work related to pairwise point cloud registration and introduces the paper’s contribution. Section 3 gives a detailed description of the proposed pairwise point cloud registration method. The proposed method is validated in experimental studies in Section 4. Finally, the conclusions and future research directions are presented in Section 5.
Normally, a pairwise coarse registration method has the following procedures. First, key primitive elements (e.g., points, lines, and planes) are detected from each point cloud, where the point-based methods are more popular due to their feasibility to different scenes[5]. The keypoint detectors (e.g., local surface patches[9], 2.5D SIFT[25], 3D SURF[24] and 3D Harris[33]) are exploited to extract keypoints from raw point clouds. Second, the feature descriptors (e.g., Spin image[22], Fast Point Feature Histograms (FPFH)[31], Viewpoint Feature Histogram (VFH)[32] and Rotational Projection Statistics (RoPS)[18]) are calculated to encode the local shape information of each keypoint. Recently, [13]
applies a deep neural network auto-encoder to realize the same effect. Third, various feature matching strategies are applied to determine the initial correspondences (
e.g., reciprocal correspondence[29], correlation coefficient[23], and chi-square test[44]). Finally, due to the fact that some of the obtained correspondences are incorrect, outliers need to be filtered from the initial correspondences, calling for a method to eliminate outliers and calculate the transformation between point clouds based on the remaining correspondences. This can be solved by some robust transformation estimation algorithms (
e.g., Random Sample Consensus (RANSAC)[14] , 3D Hough voting[37], geometric consistency constraints based on line distance or triangle property[36][41], and Game Theory based matching algorithms
[2][43].)Additionally, there are some point-based methods which do not follow the abovementioned workflow. For example, the 4-Points Congruent Sets (4PCS)[1] and its variants (e.g., SUPER-4PCS[27], Keypoint-based 4PCS (K-4PCS)[35])determines the corresponding four-point base sets by taking intersection ratios of these four points instead of using feature descriptor for matching, thus improving the efficiency of RANSAC based global registration to a great extent.
Though most of the coarse registration algorithms can generally provide satisfactory registration results, they still have limitations. For symmetric and large-scale point clouds, a robust and efficient correspondence matching algorithm, which is capable of acquiring the globally optimal correspondences, is urgently needed.
As for fine registration, the Iterative Closest Point (ICP) algorithm [7] and its variants[3][16] are the most commonly used methods, which alternate between correspondence matching and transformation calculation until convergence. [3]
proposed a Geometric Primitive ICP with Random sample consensus (GPICPR) in which the local surface normal vector and geometric curvature are used for matching and neighborhood searching.
[16] used geometric features to improve the classical ICP algorithm. ICP and these variants are able to acquire registration results with high efficiency and accuracy. However, they require a good initialization to avoid converging to bad local minimum.To overcome the limitations and challenges, this paper proposes an Iterative Global Similarity Points (IGSP) algorithm to realize a coarse-to-fine integration solution to pairwise 3D point cloud registration. IGSP is inspired by the classical ICP algorithm with two improvements of hybrid metric spaces (e.g., BSC feature and Euclidean geometry spaces) and globally optimal correspondences matching.
Specifically, the main contributions of the proposed method are as follows: i) we formulate the correspondence matching task as an energy function, which models the global similarity of keypoints on the hybrid metric spaces of BSC feature and Euclidean geometry, to get a more robust result . ii).we realize a coarse-to-fine registration by conducting optimal correspondences matching and transformation calculation in an iterative and mutually reinforcing manner, so that a good initialization is not essential.
An overview of the proposed IGSP algorithm is shown in Fig.1. Similar to ICP, an iteration process is also involved. IGSP iteratively conducts globally optimal correspondences matching and transformation calculation, until the transformation is negligible. The workflow of IGSP is shown in Fig.1a. First, we detect the keypoints of point clouds and generate the Binary Shape Context (BSC)[12] descriptors to encode their local features. Then, we formulate the correspondence matching task as an energy function, which models the global similarity of keypoints on the hybrid metric space of BSC feature and Euclidean geometry, as shown in Fig.1d-e. Next, we determine the globally optimal correspondences by optimizing the energy function by the Kuhn-Munkres(KM) algorithm[39] and then calculate the transformation based on them, as shown in Fig.1f-g. Finally, we refine the transformation between two point clouds by iteratively conducting optimal correspondences matching and transformation calculation to realize a coarse-to-fine registration under an unified framework, as shown in Fig.1h-i. The pseudo code of IGSP is shown in Algorithm 1.
In this section, a global energy function for keypoints matching is constructed, as Eq.1, which consists of data cost and penalty cost. Data cost represents the global similarity of matched keypoints in source and target point clouds, while penalty cost indicates the number of keypoints without correspondence. For data cost, and represent a matching keypoint pair. is the set of correspondences between two point clouds and means the compound distance between keypoint and ,which is defined as a weighted sum of feature distance and Euclidean distance , as shown in Eq.2. and are the weights of and . As shown in Eq.3, is defined as the Hamming distance between the BSC descriptors of and . is calculated as a scale factor related to point density times the Euclidean distance between and , as shown in Eq.4. and are assigned as Eq.5, in which is the iteration number counted from zero and controls the weight changing rate. For penalty cost in Eq.1, is the set of unmatched keypoints in two point clouds and is the number . is the weight of penalty cost, which will be further used as the criterion for mismatch judgement in 3.3.
(1) |
(2) |
(3) |
(4) |
(5) |
By minimizing the energy function, we can obtain the globally optimal correspondences , as shown in Eq.6.
(6) |
In this section, the global energy function Eq.6 is generalized into a bipartite graph minimum weight match problem and then solved by using slacked KM algorithm.
A bipartite graph is a graph whose vertices can be divided into two disjoint sets and [15]. Each edge connects a vertex in to a vertex in in this graph. Given a bipartite graph and its corresponding edges, the optimal weight matching guarantees that each node in one sub-graph can be matched to only one node in the other subgraph, and this matching can achieve global matches with the maximal or minimum summation of edge weight.
The task to find the best matching for two keypoint sets can be modeled as an optimal matching task of bipartite graph. In the weighted bipartite graph , each keypoint in source and target cloud is represented by one node respectively in set and . Suppose there are and keypoints detected from source and target cloud respectively, and when , we add virtual unmatched nodes to the set with less nodes to make . Each edge corresponds to a distance between node in and node in . With as the mismatch threshold for , is computed as follows:
(7) |
Next, the sum of all edge weights can be minimized as in Eq.6. When , there is only a constant difference between and the minimum energy for the energy function, so that the minimum weight match is equivalent to the optimization of energy function. The selected edges whose weight make up the optimal matching set , and the unmatched keypoints set is also determined.
(8) |
Given such weighted bipartite graph , the Kuhn-Munkres(KM) algorithm is employed. It outputs a complete bipartite matching with minimum matching weight by transforming the problem from an optimization problem of finding a minimum weight matching into a combinatorial one of finding a perfect matching. For efficiency concern, we apply the KM algorithm with slacked terms whose time complexity is , in which is the number of vertexes of the graph. Fig.1 illustrates a simple example of this process.
It is proved that given a point cloud pair with and keypoints respectively and a fixed threshold or , can be calculated by solving via KM algorithm.
Once the correspondence is determined, the optimal transformation can be estimated via Singular Value Decomposition(SVD), as shown in Eq.
9 .(9) |
Although a correspondence set with global similarity can be solved efficiently, it is not robust enough. Inspired by ICP, we propose an iteration process by alternating between solving for the correspondence using the method presented above and estimating the transformation using SVD until convergence. For each iteration, and are updated according to Eq.5. In the iteration process, decreases from 1 to 0, while increases from 0 to 1. At first, and the correspondence is estimated only based on keypoints’ feature descriptor, thus producing a coarse registration result. As the process continues, the weight of Euclidean distance increases, introducing the geometric restriction and refining the registration over and over.
As mentioned before, (or ) is the weight of penalty cost used as the criterion for mismatch judgement. In the context of bipartite graph minimum weight match, candidate pairs whose would be regarded as mismatches. A self-adaptive scheme for the determination of this subtle parameter is devised, as shown in Eq.10.
(10) |
When , given a pairwise match matrix of , the mean value
can be calculated. With a threshold parameter , we get the initial . When , and are the average Euclidean and feature distance of matched keypoints pairs of last iteration. Besides, and are two threshold parameter to be determined. The search space of candidate match keeps narrowing under the restriction of the average geometric and feature similarity of last iteration’s correspondence set. Generally, the larger parameters are, the larger the search space of candidate match is.Finally, as for the condition of convergence, we set thresholds for both translation and rotation in practice. Once the transformation difference between two iterations meets the condition, the iterative process stops.
Experiment Platform. The experiments are implemented with a 16 GB RAM and an Intel Core i7-6700HQ @ 2.60GHz CPU. The proposed IGSP algorithm and the compared baselines are all implemented in C++ with the help of point cloud library (PCL) [30].
Datasets description. As shown in Fig.2, the performance of the proposed IGSP algorithm is evaluated both on small scale scans of 3D models (Stanford Repository[10]) and on challenging real-world TLS datasets (e.g. Park, Indoor). These TLS datasets are challenging as (i) each of the datasets contains several scans and billions of points (ii) repetitive, symmetric and incomplete structures are common.
Evaluation criteria. We evaluate the performance of the proposed IGSP method in terms of rotation error and translation error which are commonly used for the evaluation of point cloud registration[18]. Given the estimated transform and ground truth transform , rotation error and translation error can be calculated as Eq.11,12.
(11) |
(12) |
Besides, the keypoint correspondences’ quality can be evaluated with precision and recall as Eq.13, where is the number of true positive correspondences, is the number of false positive ones, and is the number of false negative ones.
(13) |
Parameter settings. To get the result of correspondence grouping step with relatively high recall and precision, the parameters should be set reasonably. As efficiency is another important concern, parameters that result in less total iteration number would be preferred. Experiment results on three datasets are shown in Fig.3, from which IGSP’s main parameters are determined in consideration of the aforementioned criteria. Table 1 shows the parameter settings of the proposed IGSP method. As for the denotation, is the Non-maximum suppression radius of BSC. The parameter settings are used for all the experiments in this paper.
Parameter | Description | Value |
Scale factor of ED | ||
m | Iterative weight changing rate | 8 |
Initial threshold parameter | 2.5 | |
Threshold parameter for ED | 1.5 | |
Threshold parameter for FD | 1.25 |
Registration results. Fig.4 show different phases of registration results using the proposed IGSP method on three testing datasets respectively. As seen in these figures,by the IGSP method, the point cloud pairs are iteratively converged from coarse to fine and get registered successfully. To further test the robustness, the algorithm is applied on other challenging real-world datasets, as shown in Fig.5. These results show that the proposed IGSP method performs well for various scenes, including those with repetitive, symmetric, noisy and incomplete structures, which are quite challenging for previous methods.
Registration accuracy evaluation. Average rotation error and translation error on three datasets are reported in Table 3. The errors show that the proposed IGSP method performs well in aligning both the tiny model and the real-world datasets, with average rotation error less than 0.1 degree and translation error less than 0.1 meter, which provides a good foundation for further applications like 3D reconstruction and object extraction.
Accuracy analysis. Fig.6 shows the precision-recall curve during IGSP’s iterative process for three datasets, in which each point represents a temporal correspondence result. Since the global optimal correspondences are applied in each iteration, both the precision and recall of the correspondence increase through the process and finally exceed 0.75 on all these datasets, thus resulting in excellent registration performance.
Time performance analysis. A time efficiency evaluation of the proposed IGSP method is conducted with regard to the runtime in each step after proper keypoint detection parameters and Iterative convergence condition are set. Table 2 lists the the average iteration number , time for registration preparation , time for each iteration and the total runtime (). As analyzed before, the time complexity of KM algorithm is , in which is the key point number.
Dataset | #Iteration | |||
---|---|---|---|---|
Stanford | 3.31 | 0.53 | 15 | 11.26 |
Park | 8.12 | 5.65 | 13 | 81.57 |
Indoor | 6.25 | 3.92 | 14 | 61.13 |
Performance comparison and analysis.
Dataset | Method | (mdeg) | (mm) | T(s) |
---|---|---|---|---|
ICP[7] | / | / | 5.6 | |
Stanford | 3DNDT[26] | / | / | 5.3 |
30% | Super4PCS[27] | 74.21 | 0.17 | 10.5 |
overlapped | FM+GC[11] | 758.23 | 2.59 | 8.9 |
IGSP | 9.89 | 0.02 | 11.3 | |
ICP | / | / | 9.4 | |
Park | 3DNDT | / | / | 8.1 |
65% | Super4PCS | 205.14 | 187.65 | 38.0 |
overlapped | FM+GC | 184.56 | 414.61 | 35.2 |
IGSP | 93.74 | 85.12 | 81.6 | |
ICP | / | / | 6.2 | |
Indoor | 3DNDT | / | / | 6.0 |
70% | Super4PCS | 209.85 | / | 47.1 |
overlapped | FM+GC | 486.15 | 431.90 | 29.8 |
IGSP | 100.42 | 25.53 | 61.1 |
To further analyze the performance of the proposed IGSP method, several pairwise point cloud registration methods (ICP[7],3D-NDT[26],Super4PCS[27] and feature matching with geometric consistency (FM+GC)[11]) are selected for performance comparison using the Park dataset and the Indoor dataset. Key parameters of all the compared methods are set according to the parameter settings recommended in the original articles.
Table 3 lists the average registration errors and runtime of the compared methods on three datasets, in which ’’ means the registration failed (the value is great than 1000). It is found that ICP and 3D-NDT fail when good initial alignment or prior knowledge is not provided. Super4PCS has poor performance on point clouds with limited overlapping and too many similar structures, especially for the indoor corridor. IGSP outperforms the FM+GC since many correct matching keypoint pairs are also rejected by geometric consistency filter, which leads to a relatively low recall of keypoints matching. However, the time efficiency of IGSP is inferior to all the compared methods due to the iteration process and the time complexity of KM algorithm, which will be the main concern of our future work.
Nowadays, point cloud registration is the basis of many applications. This paper presented the Iterative Global Similarity Points (IGSP) algorithm, which iteratively find the corresponding keypoints considering global similarity and estimate the rigid body transformation to achieve coarse-to-fine pairwise point clouds registration. We validated its performance on different scenarios. Comprehensive experiments indicated that the proposed IGSP algorithm obtained good performance in correspondence precision, recall and registration accuracy. Although the proposed method provides satisfactory registration results, it is time consuming due to the iteration process and the time complexity of KM algorithm. In future work, we will try other efficient and reasonable methods (e.g. Minimum Cost Max Flow (MCMF) and Graph Cut) to solve the proposed energy function. A smooth term for geometric consistency in energy function will also be considered.
Special issue on 3D Representation for Object and Scene Recognition.
The three-dimensional normal-distributions transform : an efficient representation for registration, surface analysis, and loop detection
. PhD thesis, ?rebro University, School of Science and Technology, 2009.
Comments
There are no comments yet.