Minimum Potential Energy of Point Cloud for Robust Global Registration

by   Zijie Wu, et al.
The University of Western Australia

In this paper, we propose a novel minimum gravitational potential energy (MPE)-based algorithm for global point set registration. The feature descriptors extraction algorithms have emerged as the standard approach to align point sets in the past few decades. However, the alignment can be challenging to take effect when the point set suffers from raw point data problems such as noises (Gaussian and Uniformly). Different from the most existing point set registration methods which usually extract the descriptors to find correspondences between point sets, our proposed MPE alignment method is able to handle large scale raw data offset without depending on traditional descriptors extraction, whether for the local or global registration methods. We decompose the solution into a global optimal convex approximation and the fast descent process to a local minimum. For the approximation step, the proposed minimum potential energy (MPE) approach consists of two main steps. Firstly, according to the construction of the force traction operator, we could simply compute the position of the potential energy minimum; Secondly, with respect to the finding of the MPE point, we propose a new theory that employs the two flags to observe the status of the registration procedure. The method of fast descent process to the minimum that we employed is the iterative closest point algorithm; it can achieve the global minimum. We demonstrate the performance of the proposed algorithm on synthetic data as well as on real data. The proposed method outperforms the other global methods in terms of both efficiency, accuracy and noise resistance.



There are no comments yet.


page 1

page 3

page 8

page 10

page 11


SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences

This paper presents a novel randomized algorithm for robust point cloud ...

DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization

For relocalization in large-scale point clouds, we propose the first app...

Self-supervised Point Cloud Registration with Deep Versatile Descriptors

Recent years have witnessed an increasing trend toward solving point clo...

Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences

We present a fast feature-metric point cloud registration framework, whi...

Evaluation of Robust Point Set Registration Applied to Automotive Doppler Radar

Point set registration is the process of finding the best alignment betw...

Provably Approximated ICP

The goal of the alignment problem is to align a (given) point cloud P = ...

Non-iterative One-step Solution for Point Set Registration Problem on Pose Estimation without Correspondence

In this work, we propose to directly find the one-step solution for the ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Point cloud registration (or alignment) is a fundamental problem in computer vision and robotics. It is the task of establishing correspondences between a pair of point clouds, each residing in a different coordinate system, and subsequently minimizing the distances between the corresponding point pairs to register the two point clouds in the same coordinate system. Many application domains rely on point set alignments, such as 3D reconstruction, shape recognition, map relocalization, computer-aided medical diagnosis, and so on. In recent years, the ubiquity of 3D acquisition devices has lead to a growing interest in point cloud registration and the need for more effective, robust and efficient algorithms. However, 3D acquisition devices, especially the ones that operate in realtime, provide noisy point cloud data which makes it challenging to achieve efficient and accurate registration.

Over the past few decades, numerous methods have been developed for point cloud registration. The Iterative Closest Point (ICP) algorithm [1, 2] has been widely used for rigid registration of 3D point clouds due to its simplicity and performance. In its basic form, ICP first establishes correspondences between the nearest points of the two point clouds and then minimized the distance between the corresponding point pairs. Hence, ICP assumes that the nearest points of the two point clouds are corresponding points. This assumption means that the two point clouds must already be approximately registered. ICP computes the rotation and translation that would minimize the distance between the corresponding points. It then applies the rotation and translation to one of point clouds and repeats the process i.e. establishes correspondence between nearest points which may be different now. It iterates until the rotation and translation converges to zero. The ICP algorithm is intuitive and easy to implement in practice because of its conceptual simplicity.

However, the ICP algorithm is well-known to be susceptible to the local minima problem due to its assumption that the set of nearest points in the current iteration will be better correspondences than those in the last iteration. This assumption can easily fail when the point cloud data is contaminated with noise or has missing regions of overlap.

The non-convex nature of the registration problem makes it inherently vulnerable to the local minima problem and sensitive to satisfactory initialization on a case by case basis depending on the shape represented by the point clouds. As such, the ICP method does not guarantee a globally optimal registration and there is no effective way to automatically determine if it is trapped in a local minimum that is too far from the global optimum.

To deal with the local minima problem, Chetverikov et al. proposed the trimmed ICP algorithm [3] which allows the application of ICP to point clouds with partial overlap. Fitzgibbon et al. [4] employed a standard iterative non-linear optimizer (LM algorithm) to replace the closed-form

minimization part of the ICP. Nonetheless, the original ICP and all its variants are still sensitive to noise and necessitate good initialization of the registration to start with. Few heuristic methods 

[5] [6] have also been presented to alleviate the local minima problem. Another strategy [7] is to use coarse alignment with other methods, such as feature matching, to achieve a good initialization. However, feature-based methods are not always reliable and do not guarantee a globally optimal transformation. More critically, due to the -norm least squares [8]

, when minimization is applied, the optimizer runs even after reaching the global minimum, especially when dealing with contaminated real point cloud data. A small number of outliers may adversely affect the validity of the results. There are several methods proposed to deal with outliers 

[3] [4] [9] [10] and for computing consensus maximization (point pairs matching maximum) between the point clouds [11] [12] [13]. However, these methods are based on heuristics and incur additional computational costs.

We propose a robust global point cloud registration algorithm that does not require point correspondences. We introduce physics in an intuitive way to solve this computer vision problem to achieve very efficient registration. Similar to the feature-based strategy, we also essentially employ a coarse alignment to fulfil a good initialization. However, in contrast, our coarse alignment method is much more efficient and obviates the drawbacks of conventional methods. We coin our method as the Minimum Potential Energy (MPE) of point clouds.

We also extend our method to work at larger scales and present the MPL algorithm based on the proposed MPE theory for more efficient registration. Specifically, we propose the idea of

least criterion, allowing us to judge exactly whether the iteration is a globally optimal registration to facilitate the computation of the point cloud coarse alignment with torque and vector in the physics system. Finally, we employ a trimmed ICP method to precisely and efficiently approach the globally optimal registration. The method that we propose is optimal, guaranteed by the shape of the objects which are different from extracting a 3D convex hull 

[14]. The shape influenced by multi-points of inliers decide the results of the optimal registration.

We validate the proposed method with a standard point cloud repository and compare it to several existing methods. Our method is able to register the real urban dataset and shows exceptional robustness to noise and outliers. We also report detailed analysis of our method in terms of time complexity and accuracy. Moreover, compared to traditional descriptor-based methods such as FPFH [15] etc., the proposed method is more efficient in dealing with point clouds of surfaces that have small variations in geometric shape or in the curvature.

Fig. 1: Top row: Five template point clouds (red) are generated from a reference point cloud (blue) with varying scales of Gaussian noise. Bottom row: The reference point cloud has been correctly registered to all templates.

Ii Related Work

A large volume of work has been published on point cloud registration. A complete list is beyond the scope of this paper. However, to put our proposed method in context, we provide a brief review of local and global registration methods.

Ii-a Local Methods

The ICP algorithm [1] [2] is the most popular method for point cloud registration in the local category. Its introduction serves as an important milestone for the this line of research. Despite its many desirable properties, including simplicity, ICP implicitly requires a full overlap between the point clouds, which is a rare situation in practice. Therefore, the trimmed ICP algorithm [3] was proposed by Chetverikov et al. as an extension to ICP for application to asymmetric point clouds. Fitzgibbon et al. [4] optimized the point cloud registration function of the Levenberg-Marquardt algorithm and provided better convergence than ICP. However, without good initialization, the ICP and all its variants have high chances of getting trapped in a local minimum.

Another line of point cloud registration methods include the adoption of probabilistic models. Jian et al. [16] [17]

proposed a more robust method by representing a point cloud with a Gaussian mixture model (GMM) and formulated the ICP algorithm as a special case of minimizing the Kullback-Leibler divergence between two GMMs. Granger 

et al. [18]

introduced the expectation-maximization (EM) algorithm for registration error optimization, achieved by the probabilities of computation of the E-step and the parameters update of the M-step. The normal distributions transform the (NDT) 

[19] [20] algorithm and performs the optimization using the Gaussian models’ definition of each cell in a spatial grid. Magnusson et al. [21] experimentally showed more robust performance than the ICP in handling large transformations. Campbell et al. [22]

proposed to construct the GMM by using support vector machines. Despite the fact that the GMM-based methods 

[16] [17] [23] show more robust registration performance with poor initialization, all these methods are still rely on local search and can still get stuck in a local minimum.

Ii-B Global Methods

To address the local minima problem, a large number of global methods have been proposed. Typical methods include genetic algorithms 

[24] [25], simulated annealing [6], particle filtering [5] etc., which apply stochastic optimization to avoid local minima. These methods have alleviated the local minima problem to great extent. However, these methods still require good initialization, otherwise they must explore a large parameter space using heuristic search methods. Moreover, they still do not guarantee a globally optimal solution.

Descriptor-based methods, such as FPFH [15], key-point detection [26], and fast descriptor [7], extract local feature descriptors at some key-points. These descriptors of two two point clouds are then matched to find corresponding points. A transformation is then computed based on the corresponding points and applied to one of the point clouds for coarse alignment. This alignment is coarse due to errors in the corresponding points, however, it still serves as a good initialization to find the global optimum. Aiger et al. proposed the 4PCS method that could efficiently extract coplanar four points from the point cloud and establish the congruent coplanar four-points sets correspondences. The process of a RANSAC-based algorithm [27] also brings in the quadratic time complexity. Mellado et al. [28] proposed the super-4PCS method for reducing the time complexity to linear. Lei et al. employed the congruent differential information to search for the best correspondence and fulfilled a great coarse registration initialization efficiently. These descriptor-based methods are often effective on surfaces where the descriptors can be computed repeatability of the keypoints and uniqueness of the descriptors. These conditions are not always met in high noise levels and weak shape (texture) variations.

Some solutions aim to achieve the global optimal registration by introducing a branch and bound (BnB) approach that divides the parameter space into multiple sub-spaces and compute the bounds iteratively. Yang et al. [29] proposed the Go-ICP algorithm that combined the BnB (Branch and Bound) and ICP to approach the global minimum. This is the first effective global registration method in practice. Many follow up BnB-based methods have been proposed since then. Some of them focus on tighter bounds [30] [31] [32] [33] [29], while others [34] aim to speed-up the search itself. Although these methods aim to provide a globally optimal guarantee, all of them run, in the worst case, with exponential time complexities. Golyanik et al. [35] proposed a completely different method that uses the gravitational pull between point clouds to register them. This method formulates point cloud registration as a gravitation problem and showed good performance. However, it is still unable to approach a globally optimal registration especially in the presence of noise and outliers. Moreover, the method is extremely inefficient, requires a full-overlap between the two point clouds and cannot deal with problems such as hollow point clouds.

Sampling Ratio Average Run Time Estimation Error

Estimation Error variance

0.05 0.705879 0.010845 3.21E-05
0.10 2.889829 0.008355 2.07E-05
0.15 6.223227 0.004785 8.99E-06
0.20 11.26921 0.007065 1.63E-05
0.25 18.46324 0.005311 3.67E-06
0.30 25.43114 0.004270 3.03E-06
0.35 36.03378 0.003597 4.85E-06
0.40 47.33147 0.002728 2.75E-06
0.45 60.27947 0.003204 2.18E-06
0.50 255.3882 0.002947 2.07E-06
0.55 95.16613 0.002630 1.07E-06
0.60 115.0996 0.002449 7.83E-07
0.65 133.8049 0.001972 9.74E-07
0.70 148.7615 0.001974 1.13E-06
0.75 177.2722 0.001821 4.90E-07
0.80 194.7487 0.002077 6.49E-07

Registration results on Bunny dataset with increasing down-sampling ratio. Evaluation metric for error is


Iii Approach

Point cloud registration problem generally involves two D-dimensional point sets, a reference set and a template set. The template set is rotated and translated so that it aligns with the reference set. Let represent the template set and the reference set. We introduce a physical system into this problem to facilitate efficient registration. The system is adapted from the real world by adding a series of modifications to adjust it to the problem at hand. These modifications and assumptions are presented below:

  1. Every point is assumed to be a particle that has a mass but no volume to avoid collision issues in the system.

  2. The point cloud is considered to be a rigid body. The force between its internal points belongs to the internal force of the system and does not affect its motion.

  3. The point cloud resides in the constant in-homogeneous force field induced by the point cloud .

  4. The system does not follow the law of conservation of kinetic energy, meaning that the system is not isolated. Moreover, each position of is computed and its distribution is discrete.

  5. Potential energy at infinity is regarded to be zero.

Before introducing the proposed method, we first review two highly relevant methods, the ICP method III-A and the N-body simulation III-B. We formulate the point cloud registration problem in Section III-A, then perform the MPE problem reformulation in Section III-C. Besides, we also introduce the force traction computed in Section III-D and discuss the global minimum approximation theory in Section III-E.

Iii-a Problem Formulation

Point set registration is the problem of finding the best transformation parameters between sets where in this paper. The best transformation can be expressed as a tuple . For and , the former can be aligned to the latter as:


where , and are unknown rotation matrix and translation vector respectively. The contains an arbitrary number of outliers and represents the sampling noise. When multiple pairs of inliers exist, the correspondence tuple are constructed between . While the point pair do not belong to the existing inlier collection, the is an arbitrary non-zero vector. Given a point pair, whether or not it is an inlier must be judged by the algorithm. The sampling noise is intrinsic to the 3D scanning process and hence unavoidable. For the above formulation, we apply the distance evaluation equation:


where, denotes the optimal correspondence point for in Y, and represents the i-th residual error for . To achieve the optimal registration, the parameter should be a zero vector, and the vector should be minimized.

The ICP algorithm performs optimization by alternately applying the following two functions:


where and denote the optimal corresponding point pair. Equation 3 implements the transformation estimation and equation 4 matches the closest points.

Taking nearest neighbor points as the corresponding points makes the ICP algorithm susceptible to getting trapped in local minima. We argue that the matching/corresponding points of a global optimization algorithm should be computed globally. To achieve this, we propose the MPE method which considers all points during matching while distributing their weights.

Iii-B N-Body Simulation

The N-body simulation [36] involves formulating the dynamics of a large number of mutually attractive particles to model the astrophysical phenomena. It covers many applications for the problem of star-cluster dynamics, globular clusters, etc. The interactions between celestial bodies are governed by Newton’s law of gravity in a superimposed gravitational field. In the classical N-body problem, the celestial object contains a set of particles and induces a constant inhomogeneous gravitational field in a system taking the following form:


where is the force of the i-th celestial body and is the gravitational proportionality constant connecting the bodies in Newton’s law. denote the particle masses and represent the position vectors of celestial bodies , respectively. denotes the -norm distance.

To avoid problems, such as turning the normal aster into a black-hole celestial body, the assumptions and modifications are refined by setting a upper bound of traction. In this paper, we focus on the collisionless N-body simulation i.e. merging, splitting and masses transfer are not taken into account.

In the N-body problem, the total energy of the i-th particle is equal to the sum of the negative gravitational potential energy , kinetic energy () and some external potential . By convention, the reference location with zero gravitational potential energy is infinitely far away form any particle with mass, resulting in negative potential energy in any location of the system:


Without considering external forces, any energy dissipation can reduce the total energy to the minimum potential energy (MPE) i.e. in this state.

Iii-C MPE Reformulation

To overcome the local minimum problem, a global registration method must have two properties. One, the local matching process (e.g. based on nearest points) must be replaced with a process that is influenced globally. Two, different weights should be allocated to the points to alleviate the influence of outliers. We first propose MPE, a physics inspired procedure based on the modified N-body simulation, to search for fine registration. Unlike ICP, which relies on closest point-pairs, we use the gravitational field induced by the set of points (particles) to implement the registration.

Fig. 2: Illustration of potential energy and gravitation cross functions with respect to linear motion. (a) A red point moves along the axis D in 3D space. (b) The potential energy of the red point as it travels along the axis D. (c) The gravitation on the red point induced by particle o as the point moves along the axis D.

The physics-based method introduced above, incorporates gravitation into point cloud registration. To compare the physical parameters with the registration parameters; the physical position minimum  [37] is the best registration position. Figure 2 shows the cross function distribution schematic diagram of a single pair of points. The PE of the whole system is formulated as:


Where, denotes the Euclidean distance between points and represents the related vector. The PE of the system follows the fifth assumption i.e. it is zero at infinity. Take a point pair as an example and put in place of point o in Figure 2. Figure 2(a) shows (red point) traveling in space through (original point) along a straight line D, where D as a single one-dimensional transverse axis. Figure 2(b)(c) show the function of PE and the traction force as the distance between the point pair changes. Note that the limits of the PE and traction force are crossed simultaneously. When the Euclidean distance

approaches zero, the PE approaches infinity and at this moment, the point pair

is also registered.

In terms of capturing the mutation of function 7, we can easily approach the mutation point using our method. After the problem reformulation, the original point cloud registration problem is transformed into parameter searching of the traction force mutation point. To summarize, the optimal registration point pairs induce the minimum potential energy of the two point clouds system. The minimum potential energy point pairs are the best correspondence and their sets consist the optimal registration of the two point clouds.

Iii-D Criterion and Algorithm Implementation

It is easy to see from (3) that most current registrations are optimized by minimizing the distance function . However, it is well-known that the -norm least squares is not robust as it is sensitive to outliers. To make matters worse in the specific case of point set registration based on -norm optimization, incorrect matching pairs receive more weight given that their Euclidean distance is very high. Such pairs should be assigned lower weights.

To address the above problem, based on the MPE method of the N-body simulation, we propose a new evaluation that distributes appropriate weight as the Euclidean distance rises and minimizes the following error


where denotes the per-point residual error of , and and represent the optimal correspondence point pair. Given and , the criterion allows outliers in data set, and hence improves the robustness of the proposed registration method. guarantees a lower bound single point pair, precluding the registration from the incalculable infinitesimal singularity. K is a user defined constant which adjusts the weight of the point cloud and can be adaptive with additional prior information such as density and curvature.

The traditional

distance method in ICP uses a non-linear loss function

of the point-pairs to distribute weights at different distances. This loss function generally works but fails in the presence of noise and outliers. Unlike our method, its quadratic weight biases the registration. Our proposed criterion easily ignores the noises and distributes lower weights to points that are far. Figure 3 shows a toy 2D example to illustrate the difference in alignment that results from the and criterion. The left image is the registration result employing minimization, and the alignment procedure stops when the global optimal is approached. The right image denotes the -norm least square minimization where the pair of outliers lead to an erroneous registration. optimization is sensitive to changes of point pair residual error and reduce as the point pair error rises (zero in infinite distance).

Fig. 3: 2D illustration of the difference between and criteria. Two point sets (blue and red) with one outlier pair are to be registered. Notice that the registration (left) is not effected by the outlier pair of points whereas the registration (right) is effected.
Input: Point Cloud P, Q
Output: Rotation matrix R, translation vector t
1 init flags , and steps , ;
2 while  AND  do
3       , ;
4       compute , ;
5       if  then
6             ;
7             if  then
8                   ;
10             end if
12       end if
13      Transform Q with ;
14 end while
Algorithm 1 Minimum Potential Energy (MPE) Algorithm for traction force to optimal registration
Fig. 4: Illustration of MPE the rotational torque and gravitational vector

computation in 3D Cartesian coordinate system. Points A, B, C are in a same point sets and A is specified as the center. The blue ellipse denote a circle with the length of AB as the radius and A as the center in 3D coordinate, same as the black circle.

, are the gravities and direction from B and C to original point o respectively. (a) The gravitational vector computation, composed by all gravitational component (for instance, , the component of force in the direction of AB). (b) The rotation torque computation, composed by all rotation torque component (for instance, , computed by the tangential component of the and perform a cross-product with vector AB).

Algorithm 1 details the steps of the proposed MPE registration. From Section III-C, we converted the conventional point cloud registration problem to the zero-point saltus step parameter search for the traction force. The traction force is defined as


where, and denote the masses of -th and -th points from the related and point clouds respectively. The symbols and are the absolute coordinates of the points and , respectively. Moreover, is the normalized vector that point particle to particle . Every single point is regarded as a particle i.e. each point has ideal conditions-having mass but no volume or shape. Using as an abbreviation representing , produces the infinite value of the traction force . However, if the distance between the two points becomes infinitesimally small, it will trap the entire point cloud in a singularity, akin to a black hole. To avoid this situation, we introduced a non-zero real number that produces a minimum traction force to realize an extreme accumulative value. The modified equation is then


which allows us to promote the whole point cloud from a single pair of point sets. is a hyper-parameter to control the point cloud density distribution. For capturing the mutation, a pair of flags is set, and , to be read to observe the changes of rotational torque and the gravitational vector, respectively. Furthermore, to simplify calculation, we take a point C (could be an arbitrary point but usually is the center of the point cloud) and use to represent the unit vector direction from V to point . The choice of C influences the time of convergence. The rotational torque and gravitational vector are computed as:


Where, denotes the rotational torque and represents the gravitational vector. In each iteration, we recompute the and store its last values. The flags are defined as follows: , . When the flag values are less than zero, we assume that the point cloud has crossed the mutation point i.e. the force direction has reversed.

Figure 4 illustrates the derivation procedure using only a dual point system for ease of understanding. The figure shows the rotational torque and the gravitational vector computation of point B and C, under the gravitational field of point O. Point A is picked to be the computation center (the position of center C discussed above). There are two planes, defined by points ABO (the blue circle) and point ACO (the gray circle). and are the gravities generated by the traction from O to B and C. and denote the component of and in the axial direction that are then combined into the total gravitational vector . The rotational torque is computed in a similar way.

Fig. 5: Flow chart of our proposed algorithm (MPL). The vectors T and S denote the rotation torque and the gravitational vector respectively. When the condition for breaking the loop is satisfied, we employ the ICP algorithm to achieve fast descent to the global optimal minimum.

Iii-E Global Minimum Approximation Theory

The proposed MPE is a global registration method. In Section III-C, we showed that the MPE mutation point reaches the position of the best point cloud registration. A limitation of MPE is that its time complexity exponentially rises with the number of points in the point cloud. In this Section, we extend the MPE theory and propose MPL to handle larger scale point clouds.

For scalability to large point clouds, we propose a strategy for down point cloud sampling. One of the kernels for our point cloud registration method is the center of mass. Statistically, random down sampling does not change the gravitational field distribution of the raw point cloud and still reduces time complexity. We down sample the raw point sets to get the sparse sets and . The new sparse point cloud provides the same distribution as the original one but with smaller intensity gravitational fields. Given the same gravitational field, optimal global registration is guaranteed.

Whereas the down sampling scheme increases the scalability of our method to larger point clouds, it is unable to achieve perfect global minimum through the MPE method alone. Therefore, once the two point clouds are registered with the MPE method, we apply the iterative closest point algorithm to achieve a fast descent to the global optimal registration in the local convex function. Note that such a course to fine approach is applied by all registration methods. However, these methods almost always rely on feature descriptors. On the other hand, the proposed framework for global minimum approximation, as showed in Figure 5, allows the point cloud registration procedure to cross local minima and directly approach the local convex optimization function under the natural force, without the need for feature descriptors.

Iv Experiments

We implemented the proposed method on a PC with Intel i5 3.4Hz processor and designed a series of experiments to test its performance on different data sets. The evaluation metrics are time complexity and estimation error. First, we verify the theoretical properties of our method using the Stanford 3D scanning repository which contains information such as transformation, overlap ratio and ground truth. Next, we compare the proposed method with existing algorithms. Finally, we discuss the influence of different hyper-parameter settings on our method. Specifically, we compare the time complexity of the proposed MPE method and its extension to MPL.

Iv-a Experiments on Synthetic Data

In this section, we demonstrate the robustness and convergence properties of the proposed method and compare the results with existing algorithms.

Fig. 6: Estimation error versus Gaussian noise. The reference set is the Stanford bunny and the template set is randomly transformed and added with 1% to 15% Gaussian noise.
Fig. 7: Registration time versus outliers. The reference set is the Stanford bunny, and the template set is randomly transformed and added with 0 to 3000 outliers generated from a uniform model.

1) Robustness of the Proposed Algorithm: We compare our method to the Iterative Closest Point (ICP) [1], Coherent Point Drift (CPD) [38], LM-ICP (Levenberg-Marquart ICP) [4] and EM-ICP (Expectation Maximization ICP) [18] algorithms using the Stanford bunny point cloud data. Specifically, the bunny_zipper_res3 (1889 points) was chosen for this experiment. We gave a random translation and rotation (angle

) and then introduced Gaussian noise uniformly distributed and added in to template set Y. The Gaussian noise was set from 1% to 15% per unit length.

Figure 6 shows that the proposed MPL algorithm outperforms the CPD [38], LMICP [4] and EMICP [18]. We can see that with increasing levels of noise, the registration error of our method remains the lowest and the slope of the curve also remains relatively small. A visual illustration of the registration by our algorithm in the presence of noise is given in Figure 1

. The blue point set represents the original point cloud, while the red point set is generated with different levels of noise. The translation of every point from the original position is randomly computed by a Gaussian model in 3D space. The standard deviations of the Gaussian distribution are

, , , and . As shown in Figure 1, our algorithm achieves the lowest error among all algorithms - lower than others by at least . More interestingly, our method shows less error accumulation for all Gaussian noise levels.

Fig. 8: Registration results of the proposed method with varying levels of outliers. Fist Row: Reference point cloud (blue) and template point clouds (red) generated from the reference with varying levels of outliers produced by a uniform model in a cube. Second Row: Registration results of the reference point cloud and temple point cloud. For better visualization, we set the registered point pair color to green.

Figure 7 compares the estimation error of our method to CPD and LMICP with respect to increasing number of outliers (uniform noise). The outliers ( points or times the original points) were generated uniformly in a cube encompassing the point cloud data. The proposed MPL algorithm performs exceptionally well and outperforms CPD and LMICP. Our method shows robustness to a large number of outliers by taking advantage of the global PE minimum to ignore the gravitational field induced by the outliers. The estimation error always stays at a low value approaching zero in this experiment irrespective of the number of outliers. Figure 8 visually demonstrates the registration effectiveness with increasing number of outliers. The original point set is coloured blue, and the red point set is generated by adding random outliers in the data cube following a uniform distribution. The number noise points are , , and . For better visualization, we show the registered model (inliers - with distance) in green.

Fig. 9: The time cost of the different model sizes is found by applying multiple methods. The left six data points are the comparison between the four methods with points of the model size (from 1000 to 7500 points). The rest of the data points show the performance of our method (MPL and the MPE process of the whole algorithm), while the other method cannot handle it.

2) Time to Convergence: The next experiment is designed to show the time complexity as the model size number is raised. We time the entire running procedure and visualize the results, as in Figure 9. The CPD and EMICP are used to compare to our method to illustrate the time complexity performance of our approach. CPD demonstrates the biggest computational cost that rises exponentially. EMICP shows high time complexity at the beginning of the pic, and medium performance in terms of model size increasing. The computational complexity of the two methods above increase exponentially, and the maximum size of the test model is limited within 7500 points. We down-sample the original bunny_zipper (35947 points) to a list of smaller models that point the number growing from 1000 to 35000.

A comparison is shown by limiting the model size range within 1000 to 7500. Beyond that, we test the bigger model size on our proposed algorithm individually. Specifically, the blue line represents the time cost of down-sampled MPE procedure in MPL algorithm.The single MPE procedure can run in low time complexity. This causes the centroid capture to need only a few points that maintain the simple shape, and the increasing the model points just make the centroid calculations more approximate to the ground truth, exerting little influence on the global minimum approach. The complete method also shows great performance and low time complexity changes with a lower rate of increase.

Fig. 10: The results of the estimation error and convergence time with different sampling ratios. The every blue inverted triangle denotes a data point that represents the relation of estimation error and convergence time, under specific sampling ratios. The red line represents their averages and the variance of estimation error with increasing sampling ratio. The metrics are calculated with 160 runs for 16 different sampling ratios
Fig. 11: Registration results of the experiment with varying sampling ratios. The first column: Reference point cloud (top red point set) and temple point cloud (bottom blue point sets). The rest of the figures are the registration results varying with increasing sampling ratios, from top to bottom and left to right.

To register point clouds with our proposed method, a down-sampling rate is generally required. A higher sampling rate usually not only brings us higher accuracy but also more runtime. Therefore, to maintain the balance between the down-sampling size of MPE computation and algorithm accuracy, we designed an independent experiment to obtain 160 data points of multiple test results from experiments repeated 10 times. In each experiment, we set the down-sampling rate from 5% to 80% with an interval of 5%. Then we tested the time to convergence and the error for each ratio of the down-sampling. Everything mentioned above is visualized in Figure 10. The blue inverted triangle points are the original test data point, whereas the red line represents the time average of each down-sampling and its corresponding standard deviation. Obviously, as the sampling ratio increases, the estimation error becomes smaller and more stable. The significant point is that the error line becomes smoother when the sampling ratio value is over 0.25. The experiment also shows that another threshold is the 0.15 sampling ratio. Once the value is bigger that 0.15, our algorithm can ensure a global optimal more frequently. More detail is showed in Table I, and the two threshold point data are thickened. The more intuitive demonstration of the sampling ratio change is also visualized in Figure 11. The first column shows the original point cloud, in which the red point set represents the temple set Y and the blue point set is one that has been given a random translation and rotation. Except for this column, the next eight images follow a rule that sampling rate increases progressively from left to right and from top to bottom.

NM MPE MPL (200) MPL (100)
4533 16.4s 3.3s 0.9s
18893 5.1m 3.9s 1.5s
81713 1.6hr 7.2s 4.8s
359473 30.6hr 20.2s 18.4s
TABLE II: The Time Cost of Minimum PE Registration For The Naive MPE Theory and Two Down-Sampling Rates of The MPL Algorithm

We also tested the performance between the original MPE methods and the modified strategy MPL with a large number of points. Four bunny sets of sizes in Stanford 3D point cloud repository were used: 4533, 18893, 81713 and 359473. For the down-sampling parameters, we tested the two most common values used in our algorithm: 100 points and 200 points. Table II shows the registration time with MPE and MPL. The MPL algorithm was significantly faster and showed a low registration time growth, no matter the number of sampling points chosen. Furthermore, through comparing the two MPLs (MPL 200 and MPL 100), we noted that the MPE procedure of the fewer points was less time-consuming. Therefore, we concluded that the PE theory is suitable for coarse registration with low time cost to fulfill a more effective point cloud registration.

Iv-B Experiment on real data

1) Performance Comparison on UWA Dataset: We firstly demonstrate the performance of our proposed registration algorithm on the UMA 3D Modeling dataset [39] [40]. These dataset is acquired with a Minolta Vivid 910 scanner and contains several images scanned from four objects. We fulfill the registration of different views by processing the raw point clouds of four objects. Due to the absence of ground truth, we apply the rotation error and translation error to measure the performance of registration result. The rotation error and translation error are calculated as following:


Where the , represents the ground truth rotation and translation between the i-th and j-th point sets. The point clouds are coarse aligned manually and then refined with ICP algorithm for the ground truth generation. The , denotes the estimation rotation and translation respectively.

We perform the comparison of rotation error in Table III with different methods including Rotational Projection Statistics (RoPs) [41], Trimmed-icp [3] and the combination RoPs+generalized-icp [42] [43]. The RoPs and its related RoPs+generalized-icp algorithms are input with points coordinate and mesh to make use of more information for registration. On the contrary, the trimmed-icp and our proposed method are only input with point coordinate. Furthermore, Our proposed method still achieve the best performance of registration results. Due to the difference of mesh resolution, we report in Table IV the translation error comparison with the trimmed-icp.The demonstration of the registration results on UWA dataset is showed in Figure 12. The first and third columns show the registration initialization. while the others represent the aligned results. In this experiment, we compare the performance our proposed method with several methods on UWA dataset. The results show that our algorithm achieves the best performance even input with fewer data information.

Input Data Chef Chicken Parasaurolophus T-Rex
RoPs [41] xyz+Mesh 2.2117 1.0075 1.0634 1.3722
RoPs+generalized-ICP  [42] [43] xyz+Mesh 0.2712 0.3900 0.1771 0.3758
Trimmed-icp [3] xyz 5.3751 5.0682 1.9188 1.6655
Ours xyz 0.2651 0.3865 0.1527 0.3230
TABLE III: Rotation Error (deg) of Registration Results of Four Individual Objects on UWA Dataset
Trimmed-icp Ours
Chef 6.9117 0.5775
Chicken 0.8773 0.4753
Parasaurolophus 7.1050 0.3031
T-Rex 3.9467 0.4006
TABLE IV: Translation Error of Registration Results of Four Individual Objects on UWA Dataset. Evaluation Metric is the Distance.
Fig. 12: The demonstration of registration with our method on the UWA DataSet. Column 1,3 denote the initialization and column 2,4 represent the aligned images. The red images denote the reference sets and the green images represent the template sets transformed with a rotation and translation.

2) Specific Blade Point Cloud Registration: This experiment shows the ability of dealing with specific point clouds. Due to the local features of the neighboring points being extremely similar, extracting the feature of the surface object showing a complex special-shaped surface (example, aero-engine blade) is difficult. In this situation, the traditional descriptor-based method [1] [7] [15] is not useful. We demonstrate the comparison of the performance between our proposed algorithm and the traditional descriptor-based method, FPFH. The point cloud data is scanned and processed from the aero-engine blade.

Figure 13

shows the effectiveness of the registration between the two methods. The original data is showed as the red point set and the green set is translated and rotated from the original point cloud. Furthermore, the blue line connects the point pair of the inliers. The right image shows the registration point pair of the FPFH method using feature extraction, and ours is showed on the left. For a more intuitive visualization, the point pairs of the inliers are partially displayed. With our method, the point pair is considered as the correspondence, especially when the distance of it is less than a certain threshold. However, when dealing with the blade surface point cloud, the FPFH method tends to fail. Moreover, the correctness matching rate is less than 15% with the whole blade, whereas the matching accuracy of the surface part is less than 4%. The precision of the matched is defined as:


Our proposed method registers the point cloud without relying on a feature-based theory. It is significant to deal with complex, special-shaped surfaces that lack the texture of information, and the method has proven its superiority.

Fig. 13: The comparison of the registration results between our proposed method and the FPFH (the descriptor-based method). Data: Reference point cloud (red point set) and temple point cloud (green point sets). The blue lines connects the registered point pair. For better comparison, the conception that the point pair registered by our method (left figure) is assumed as a correspondence. And the FPFH correspondence point pair is the right-side figure.
Fig. 14: The demonstration of real the street data registration results. Four scenes in [44] are selected to perform the ability of large scale shape feature extraction. (a) shows the high overlap urban lidar street data registration. On the contrary, (b) is only with few overlap on cross shape feature. The red,blue point sets denote the reference sets and the template sets respectively.

3) City lidar Point Cloud Registration: Finally, we conduct a test on city lidar data using our proposed method of urban registration. The CITY scans were collected by Alastair Quadros in 2013, and contain a variety of common urban road objects with wide-baselines, which is quite difficult to reconstruct. The testing point clouds were extracted from Sydney’s urban objects dataset [44]. We selected several scenes of long streets to demonstrate the registration with large scale global shape feature extraction, whereas the traditional methods (such as icp, FPFH, and etc.) can hardly achieve it.

Here, we register the urban road point cloud and shown in Figure 14. Figure 14(a) shows a high overlap condition (the shape overlaps more than 80%) of the real road lidar point cloud registration and displays a fine performance. Another situation that contains a lower overlap and less information retained except, basically, the shape, is shown in Figure 14(b). It is clear that our method has a fine performance in urban road point cloud data application and can easily register the global feature, for instance, the two street and a cross in common urban scenes.

Real city lidar data contains a wide scale sparse point cloud, while the two scenes point clouds hardly overlap. The ground truth of the point pair are not known, and the truth correspondence is biased due to the wide scale scan procedure. To deal with the different situations, since the traditional method relying on the high accuracy of the source point cloud tends to fail, our method introduces the different weights of different parts generated by the density of the points at different locations. The proposed method can extract the global shape feature to fulfill a more accurate registration, which is the most important application of our theory.The dense clusters present more information, and our method distributes more weight to the kind of cluster to obtain a fine registration result. The results show that our method performs great within a large scale city lidar point cloud; the proposed method still stays efficient and accurate.

V Conclusion

In this paper, we proposed a novel, globally optimal theory for point cloud registration in 3D, which was achieved by introducing a specific physical system to approach the MPE of the point clouds. We also proposed a new criterion for the over-fitting problem. Our method of gathering the point clouds through rotation torque and gravitational vector fulfilled the globally optimal registration efficiently. The proposed approach was able to align point clouds within large quantities of Gaussian and uniform noise, while running with low time complexities. We showed a fine performance with a comparison between our proposed method and some other approaches. Besides, we discussed the drawbacks of traditional descriptor-based methods and explored a new path to offer a new solution. The proposed approach was also able to register a variety of common urban road objects.


This work was supported in part by National Natural Science Foundation of China under Grant 61573134, Grant 61733004 and in part by the Australian Research Council under Grant DP190102443.


  • [1] P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures, vol. 1611.   International Society for Optics and Photonics, 1992, pp. 586–606.
  • [2] Y. Chen and G. Medioni, “Object modelling by registration of multiple range images,” Image and vision computing, vol. 10, no. 3, pp. 145–155, 1992.
  • [3] D. Chetverikov, D. Stepanov, and P. Krsek, “Robust euclidean alignment of 3d point sets: the trimmed iterative closest point algorithm,” Image and vision computing, vol. 23, no. 3, pp. 299–309, 2005.
  • [4] A. W. Fitzgibbon, “Robust registration of 2d and 3d point sets,” Image and vision computing, vol. 21, no. 13-14, pp. 1145–1153, 2003.
  • [5] R. Sandhu, S. Dambreville, and A. Tannenbaum, “Point set registration via particle filtering and stochastic dynamics,” IEEE transactions on pattern analysis and machine intelligence, vol. 32, no. 8, pp. 1459–1473, 2009.
  • [6] C. Papazov and D. Burschka, “Stochastic global optimization for robust point set registration,” Computer Vision and Image Understanding, vol. 115, no. 12, pp. 1598–1609, 2011.
  • [7] H. Lei, G. Jiang, and L. Quan, “Fast descriptors and correspondence propagation for robust global point cloud registration,” IEEE Transactions on Image Processing, vol. 26, no. 8, pp. 3614–3623, 2017.
  • [8] K. MacTavish and T. D. Barfoot, “At all costs: A comparison of robust cost functions for camera correspondence outliers,” in 2015 12th Conference on Computer and Robot Vision.   IEEE, 2015, pp. 62–69.
  • [9] S. Rusinkiewicz and M. Levoy, “Efficient variants of the icp algorithm.” in 3dim, vol. 1, 2001, pp. 145–152.
  • [10] G. Champleboux, S. Lavallee, R. Szeliski, and L. Brunie, “From accurate range imaging sensor calibration to accurate model-based 3d object localization,” in

    Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition

    .   IEEE, 1992, pp. 83–89.
  • [11] P. Speciale, D. Pani Paudel, M. R. Oswald, T. Kroeger, L. Van Gool, and M. Pollefeys, “Consensus maximization with linear matrix inequality constraints,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4941–4949.
  • [12] Y. Liu, C. Wang, Z. Song, and M. Wang, “Efficient global point cloud registration by matching rotation invariant features through translation search,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 448–463.
  • [13] H. Yang and L. Carlone, “A polynomial-time solution for robust registration with extreme outlier rates,” arXiv preprint arXiv:1903.08588, 2019.
  • [14] J. Fan, J. Yang, F. Lu, D. Ai, Y. Zhao, and Y. Wang, “3-points convex hull matching (3pchm) for fast and robust point set registration,” Neurocomputing, vol. 194, pp. 227–240, 2016.
  • [15] R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms (fpfh) for 3d registration,” in 2009 IEEE International Conference on Robotics and Automation.   IEEE, 2009, pp. 3212–3217.
  • [16] B. Jian and B. C. Vemuri, “A robust algorithm for point set registration using mixture of gaussians,” in Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 2.   IEEE, 2005, pp. 1246–1251.
  • [17] ——, “Robust point set registration using gaussian mixture models,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 8, pp. 1633–1645, 2010.
  • [18] S. Granger and X. Pennec, “Multi-scale em-icp: A fast and robust approach for surface registration,” in European Conference on Computer Vision.   Springer, 2002, pp. 418–432.
  • [19] M. Magnusson, A. Lilienthal, and T. Duckett, “Scan registration for autonomous mining vehicles using 3d-ndt,” Journal of Field Robotics, vol. 24, no. 10, pp. 803–827, 2007.
  • [20] T. Stoyanov, M. Magnusson, H. Andreasson, and A. J. Lilienthal, “Fast and accurate scan registration through minimization of the distance between compact 3d ndt representations,” The International Journal of Robotics Research, vol. 31, no. 12, pp. 1377–1393, 2012.
  • [21] M. Magnusson, A. Nuchter, C. Lorken, A. J. Lilienthal, and J. Hertzberg, “Evaluation of 3d registration reliability and speed-a comparison of icp and ndt,” in 2009 IEEE International Conference on Robotics and Automation.   IEEE, 2009, pp. 3907–3912.
  • [22] D. Campbell and L. Petersson, “An adaptive data representation for robust point-set registration and merging,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4292–4300.
  • [23] Y. Tsin and T. Kanade, “A correlation-based approach to robust point set registration,” in European conference on computer vision.   Springer, 2004, pp. 558–569.
  • [24] L. Silva, O. R. P. Bellon, and K. L. Boyer, “Precision range image registration using a robust surface interpenetration measure and enhanced genetic algorithms,” IEEE transactions on pattern analysis and machine intelligence, vol. 27, no. 5, pp. 762–776, 2005.
  • [25] C. Robertson and R. B. Fisher, “Parallel evolutionary registration of range data,” Computer Vision and Image Understanding, vol. 87, no. 1-3, pp. 39–50, 2002.
  • [26] A. Mian, M. Bennamoun, and R. Owens, “On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes,” International Journal of Computer Vision, vol. 89, no. 2-3, pp. 348–361, 2010.
  • [27] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981.
  • [28] N. Mellado, D. Aiger, and N. J. Mitra, “Super 4pcs fast global pointcloud registration via smart indexing,” in Computer Graphics Forum, vol. 33, no. 5.   Wiley Online Library, 2014, pp. 205–215.
  • [29] J. Yang, H. Li, D. Campbell, and Y. Jia, “Go-icp: A globally optimal solution to 3d icp point-set registration,” IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 11, pp. 2241–2254, 2015.
  • [30] T. M. Breuel, “Implementation techniques for geometric branch-and-bound matching methods,” Computer Vision and Image Understanding, vol. 90, no. 3, pp. 258–294, 2003.
  • [31] A. Parra Bustos, T.-J. Chin, and D. Suter, “Fast rotation search with stereographic projections for 3d registration,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3930–3937.
  • [32] D. Campbell and L. Petersson, “Gogma: Globally-optimal gaussian mixture alignment,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5685–5694.
  • [33] R. I. Hartley and F. Kahl, “Global optimization through rotation space search,” International Journal of Computer Vision, vol. 82, no. 1, pp. 64–79, 2009.
  • [34] X. Li, Y. Liu, Y. Wang, C. Wang, M. Wang, and Z. Song, “Fast and globally optimal rigid registration of 3d point sets by transformation decomposition,” arXiv preprint arXiv:1812.11307, 2018.
  • [35] V. Golyanik, S. Aziz Ali, and D. Stricker, “Gravitational approach for point set registration,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 5802–5810.
  • [36] S. J. Aarseth, Gravitational N-body simulations: tools and algorithms.   Cambridge University Press, 2003.
  • [37] J. R. Vinson, “Theorem of minimum potential energy, hamilton’s principle and their applications,” in Plate and Panel Structures of Isotropic, Composite and Piezoelectric Materials, Including Sandwich Construction.   Springer, 2005, pp. 121–141.
  • [38] A. Myronenko and X. Song, “Point set registration: Coherent point drift,” IEEE transactions on pattern analysis and machine intelligence, vol. 32, no. 12, pp. 2262–2275, 2010.
  • [39] A. S. Mian, M. Bennamoun, and R. A. Owens, “A novel representation and feature matching algorithm for automatic pairwise registration of range images,” International Journal of Computer Vision, vol. 66, no. 1, pp. 19–40, 2006.
  • [40] A. S. Mian, M. Bennamoun, and R. Owens, “Three-dimensional model-based object recognition and segmentation in cluttered scenes,” IEEE transactions on pattern analysis and machine intelligence, vol. 28, no. 10, pp. 1584–1601, 2006.
  • [41] Y. Guo, J. Wan, J. Zhang, K. Xu, and M. Lu, “Efficient registration of multiple range images for fully automatic 3d modeling,” in 2014 International Conference on Computer Graphics Theory and Applications (GRAPP).   IEEE, 2014, pp. 1–8.
  • [42] Y. Guo, F. Sohel, M. Bennamoun, J. Wan, and M. Lu, “An accurate and robust range image registration algorithm for 3d object modeling,” ieee transactions on multimedia, vol. 16, no. 5, pp. 1377–1390, 2014.
  • [43] J. Williams and M. Bennamoun, “Simultaneous registration of multiple corresponding point sets,” Computer Vision and Image Understanding, vol. 81, no. 1, pp. 117–142, 2001.
  • [44] M. De Deuge, A. Quadros, C. Hung, and B. Douillard, “Unsupervised feature learning for classification of outdoor 3d scans,” in Australasian Conference on Robitics and Automation, vol. 2, 2013, p. 1.