Point set registration is a fundamental and important technique in many domains, such as compute vision, robotics, and computer graphics, etc. For each range scan given in a set-centered frame, the registration goal is to find an optimal rigid transformation (global motion) and transform it into the reference coordinate frame. Due to the number of involved point sets, point set registration can be divided into pair-wise registration and multi-view registration problems. In the past few decades, lots of effective approaches have been proposed to solve the pair-wise registration problem. Among these approaches, the iterative closest point (ICP) algorithm is one of the most popular methods. Based on this basic algorithm, many ICP variants  have been proposed to improve the performance of pair-wise registration in different perspectives. For convenience, we will use the term rigid transformation and motion interchangeable throughout this paper.
Different from pair-wise registration, the multi-view registration problem is more complex and has attracted less attention. In the literature, some approaches have been proposed to solve this difficult problem. For example, Chen et al.  proposed the alignment-and-merging approach, which repeatedly aligns and merges two scans until all scans are merged into the whole model. This approach is straightforward but suffers from the error accumulation problem. To address this issue, Evangelidis et al. 
proposed the JRMPC approach, which assumes that all points are realizations of a unique Gaussian mixture model (GMM) and therefore casts the registration into clustering problem. Subsequently, the expectation maximization (EM) algorithm is utilized to estimate GMM parameters as well as all global motions for multi-view registration. This approach is time-consuming due the large number of parameters required to be estimated. Therefore, Zhu et al. introduced the -means algorithm to solve the multi-view registration problem. Compared with the JRMPC, the -means based approach is more efficient and likely to obtain better registration results.
As these approaches estimate each global motion sequentially, they are more likely to be trapped into a local minimum, especially when the scan number is large. Therefore, Krishnan et al.  proposed the optimization-on-a-manifold approach to simultaneously optimize all motions. To obtain desired results, it requires to establish accurate point correspondences for all scan pairs, which is very difficult in practice. Subsequently, Mateo et al.  extended this approach under the Bayesian perspective, which views pair-wise correspondence as missing data and solves the registration problem by the EM algorithm. Although this approach can simultaneously optimize all global motions, it requires to compute a huge number of latent variables, which is time consuming.
For multi-view registration, another feasible solution is to recover global motions from a set of relative motions. To this end, Govindu  proposed the motion averaging (MA) algorithm, which avoids averaging of motion in Lie groups but performs average in the Lie-algebra of the underlying motion representation. With an initial guess, global motions can be simultaneously recovered from a set of relative motions by MA algorithm, which was further extent to solve multi-view registration problem . Although these two algorithm is effective, it is sensitive to outliers due to utilizing Frobenius norm error in optimization. Govindu  combined graph-based sampling scheme and Random sample consensus (RANSAC) method to remove motion outliers. This approach is more robust, but the efficiency is seriously reduced with the increase of scan number. For robot mapping, Grisetti et al.  proposed the general framework for graph optimization, called as G2O, which takes the same inputs as that of MA algorithm. Similar to MA algorithm, it is effective but sensitive to outliers.
Besides, Bourmaud et al.  proposed Bayesian MA algorithm for robot mapping. It is more complex than the original MA and its performance is greatly affected by the assignment of a reasonable covariance to each relative motion, which is very difficult in real applications. Meanwhile, Arrigoni et al.  introduced the low-rank and sparse (LRS) matrix decomposition to solve multi-view registration, which concatenates all available relative motions into a large matrix and then decomposes it into one sparse matrix and one low-rank matrix. This approach can be viewed as another MA method and that is robust to outliers, but it requires more relative motions to achieve good registration. What’s more, these methods treat each relative motion equally, and this will reduce the performance of registration. Accordingly, Guo et al.  proposed weighted MA algorithm and Jin et al.  proposed weighted LRS algorithm, which can really improve the performance of multi-view registration with each relative motion assigned by a suitable weight, e.g. reliable motions assigned with high weights. However, it is difficult to manually assign a suitable weight to each relative motion.
Previous MA methods often use Frobenius norm error in optimization, and they perform well under the assumption of Gaussian noises. However, in practice, a relative motion set often includes outliers. In this case, the Frobenius norm error can not properly capture error statistics, which may seriously degrade the performance. Recently, correntropy  has been proposed as an information theoretic learning measure to solve robust learning problems [4, 5, 7, 13]
. Compared with Frobenius norm, correntropy includes all even moments of the error. Therefore, the correntropy measure is robust against outliers and can achieve better learning performance especially when data contain large outliers.
Accordingly, this paper introduces the correntropy measure to reformulate the MA problem, which is difficult to be solved directly. To this end, the half-quadratic (HQ)  technique is utilized to transform the problem into a half-quadratic optimization problem, which can be solved by the traditional optimization method. Further, we design an adaptive selection strategy for kernel width to take advantage of correntropy properties. Compared with Frobenius norm error, the negative effects of outliers are therefore alleviated by the correntropy measure. In summary, the main contributions of this paper are delivered as 1) It proposes a novel cost function for robust motion averaging. 2) It develops an effective MA algorithm by the HQ technique. 3) Experiments carried out on benchmark data sets confirm its superior performance over other state-of-the-art algorithms.
The remainder of the paper is organized as follows. Section 2 briefly briefly reviews the concepts of MCC and HQ optimization theory. Section 3 formulates the correntropy based objective function for motion averaging and proposes the HQ based algorithm. Following that is section 4, in which the proposed approach is tested and evaluated on four benchmark data sets. Finally, conclusions are drawn in Section 5.
This section briefly reviews MCC and HQ optimization theory, which are bases of the proposed approach.
2.1 Maximum correntropy criterion
Given two random variablesand , the correntropy is defined by:
where denotes a shift-invariant Mercer kernel and
is the joint probability distribution function (PDF) of. In practice, the joint PDF is unknown and only a finite number of data points are available. With finite samples , the correntropy can be approximated as:
Usually, the correntropy kernel utilizes Gaussian Kernel:
where is the kernel width and is the error term.
Obviously, the correntropy is a local and nonlinear similarity measure between two random variables within a ”window” in the joint space defined by the kernel width. Compared with traditional measures, the correntropy contains all the even moments of the difference between and
which is referred to as the MCC.
2.2 Half-quadratic optimization theory
Usually, it is difficult to directly optimize the correntropy based objective function, which is non-quadratic. Therefore, the HQ technique has been introduced to solve this problem.
According to the HQ theory , there is a convex conjugated function corresponding to and they have the following relationship:
where and the maximum is achieved at . Equivalently, Eq. (5) can also be transformed into:
By defining and , Eq. (6) can be further derived as:
Based on the HQ technique, the non-quadratic cost function is reformulated as the augmented objective function in enlarged parameter space by introducing auxiliary variable .
3 Robust Motion Averaging under MCC
This section states the MA problem in multi-view registration and then proposes a robust solution under MCC.
3.1 Problem statement
Given multiple rang scans, the goal of multi-view registration is to estimate the rigid transformation for each scan to the reference coordinate frame. For simplicity, the rigid transformation can be defined in the form of motion as:
denote the rotation matrix and translate vector, respectively. Compared with the multi-view registration problem, the pair-wise registration problem is much easier. Therefore, it is reasonable to achieve multi-view registration based on pair-wise registration, which arises the MA problem. Given a set of estimated relative motions, it requires to recover the global motion for multi-view registration. Accordingly, the multi-view registration can be formulated the following optimization problem:
As either or denotes the variable of global motion, we only preserve as the variable for the simplicity. This problem has been solved by the original MA algorithm , which is sensitive to outliers due to the application of Frobenius norm error in the optimization.
To improve the robustness, we introduce correntropy as the error measure and reformulate the multi-view registration problem as the following optimization problem:
Eq. (10) denotes a non-convex and non-quadratic cost function, which is difficult to be directly minimized by traditional methods. To this end, the HQ technique should be utilized to minimize this function.
3.2 Optimization by the HQ theory
As shown in Eq. (7), minimizing the correntropy measure based loss function in terms of equals to minimizing an augmented cost function in an enlarged parameter space . Accordingly, the correntropy measure based objective function can be further formulated as:
Further, we can define the augmented cost function:
According to the HQ optimization theory, we obtain the equivalent relation as follows:
This optimization problem can then be solved by the alternating minimization procedure as follows:
(1) Optimization of : According to Eq. (5) and Eq. (7), the minimum of the objective function is achieved by for given a certain . Therefore, the optimal solution of can be estimated for the fixed as:
This procedure can be viewed as the weight assignment operation, which assigns different weights to each relative motion based on the residual motion error. According to the property of Gaussian function, a relative motion with small error will be assigned with a large weight, and vice versa. Different from previous methods, we do not manually estimate a weight for each relative motion, but automatically calculate them by the residual motion error. Therefore, suitable weight can be assigned to each relative motion due to properties of the correntropy measure.
(2) Optimization of : For the fixed , Eq. (12) is simplified into the following optimization problem:
Eq. (15) denotes the weighted MA problem, where the negative impact of outliers can be seriously reduced due to the small weight assigned by the first procedure. Since this problem can be solved by the variant of original MA algorithm, we present the solution without any provement.
3.3 Weighted motion averaging
Given the relative motion set , the motion averaging algorithm requires initial global motions to achieve multi-view registration by iterations. For one relative motion and previous global motion , the residual relative motion is defined as:
Eq. (16) can be converted into the equivalent formulation:
where . Subsequently, the function is utilized to extract parameters from to form a column wise vector and then Eq. (17) is transformed into the following form:
As each relative motion is assigned with a weight in our approach, the block-matrix is constructed with the th and th block-elements filling with and :
where . To refine global motions, Eq. (20) can be extended to the situation of many relative motions as follows:
where and . This formulation leads to vector including parameters of all residual global motions:
where is the pseudo-inverse matrix of . Finally, elements of can be utilized to update each global motion as:
where is residual global motions and denotes the inverse function of .
Obviously, our method is local convergent. To obtain desired results, initial guess should be provided for global motions in advance. Besides, its performance is affected by the kernel width in correntropy measure. In the literature, lots of works have illustrated that relatively large can offer high convergence speed but suffer from less accuracy, and vice versa. As our approach achieves multi-view registration by iterations, it is better to use adaptive kernel width. Specifically, the kernel width is set to be large at the beginning of the iteration and it should decrease with the increase of the iteration number. As residual motion error of decreases with the increase of iteration number, it is reasonable to set the kernel width to be proportional with the residual error of all global motions, e.g. , where is a preset parameter and denotes the residual motion error defined as:
This setting can well balance the convergence speed and accuracy of the proposed method.
Based on the above description, the proposed method is summarized in Algorithm 1, where the parameter will be discussed and determined in experiments.
This section tests and evaluates our approach on five benchmark data sets, where four data sets are taken from the Stanford 3D Scanning Repository  and the Hand data set is provided by Torsello . Each of them was acquired from one object model in multiple views and ground truth of rigid transformations was provided with multiple scans for the evaluation of registration results. But they are only utilized to assist for the final assessment. All experiments are performed on a four-core 3.6 GHz computer with 8 GB of memory.
As the proposed approach takes relative motions as its input to recover global motions, we estimate relative motions for each scan pair by utilizing the pair-wise registration method proposed in , which can obtain reliable results for these scan pairs with non-low overlap percentage. Given a set of scans, many scan pairs contain low overlap or non-overlapping percentages, their estimated relative motions are unreliable and meaningless. For accurate registration, it is better to utilize as many reliable relative motions but few unreliable relative motions as possible. Therefore, we only select relative motions of these scan pairs, whose trimmed mean square errors are less than the predefined threshold. For accuracy comparison, the registration error of rotation matrix and translation vector are defined as and , respectively. Here, indicates the ground truth of the th rigid transformation and denotes the one estimated by multi-view registration approach. Table I and Fig. 1 demonstrate some details of these four data sets as well as preserved motion sets. Besides, Table II we list statistics information of all relative motions in each data set, including the mean, median, RMSE of rotation and translation errors. As shown in Table II, each preserved motion set still contains outliers.
4.1 Parameter tuning
The performance of our approach is related to the selection of kernel width , which is set to be . Empirically, we can set , which directly assigns the residual motion error to the kernel width. Here, we do experiments to find its appropriate value and check whether the performance of our method is sensitive to this parameter. More specially, we change the value of in our approach and test it on all four data sets. Experimental results are reported in the form of registration errors . During the experiment, we find the setting of around 1.0 is more likely to obtain promising results for multi-view registration. Fig. 4 records registration results of our approach with varied values of around 1.0.
As shown in Fig. 4, we can observed that: 1) the setting of tends to obtain the desired results. 2) The performance of our approach is stable as long as is set to be within a certain value range, e.g. . Accordingly, the proposed approach is robust to the parameter as long as it is chosen from a reasonable range, which makes it easy to apply this approach without much effort for parameter tuning. However, both too large and too small may result in undesired registration results. For large , the correntropy measure is difficult to discriminate outliers from all relative motions, so the proposed method is unable to recover accurate global motions from a set of relative motions including outliers. For small , inliers with small noises may be viewed as outliers, which also makes the proposed method be unable to obtain desired registration results. Considering all these factors, we set , i.e. in the proposed method in the following experiments.
To demonstrate the performance, the proposed method is tested on four data sets and compared with some related approaches, including the multi-view registration approach based on the low-rank and sparse decomposition algorithm , original motion averaging algorithm , and weighted motion averaging algorithm , which are abbreviated as LRS, WA, and wWA, respectively. It should be noted that LRS does not requires initial guess for multi-view registration, but all other three approaches require initial global motions. As only relative motions are available in each data set, the output of LRS is taken as the input of other three approaches for multi-view registration. Experimental results are reported in the form of run time, rotation error, and translation error. These registration results are all recorded in Table III. For the evaluation of registration accuracy in a more intuitive manner, Fig. 3 displays all multi-view registration results in the form of a cross-section.
As shown in Table III and Fig. 3, LRS is robust to unreliable relative motions. Without the initial guess of global motions, LRS may efficiently achieve multi-view registration by utilizing a set of relative motions. However, it requires a high proportion of available relative motions to obtain promising registration results. As shown in Fig. 1, proportions of available relative motions are all below for these four data sets, so it is difficult to obtain promising registration results due to a low proportion of available relative motions.
Similar to robot mapping, the multi-view registration can be achieved by the G2O method, which takes relative motions and some results of LRS as its inputs. Besides, each relative motion requires to be assigned with one covariance matrix to denotes its uncertainty or reliability. Here, we assign the identity matrix to each relative motion due to the lack of prior information. As Table III and Fig. 3 demonstrate, the G2O method is efficient but is unable to obtain promising registration results. To obtain the desired results, each relative motion requires one appropriate covariance matrix, which is very difficult in most of practical applications.
For multi-view registration, both MA and wMA take registration results of LRS as their initial guess. As MA utilizes Frobenius norm error for the estimation of global motions, it is sensitive to outliers and difficult to obtain promising registration results due to the exiting of outliers. Different from MA, wMA pays more attention to reliable relative motions by assigning high weights. When each relative motion is assigned with one appropriate weight, e.g. outliers assigned with very low weight, wMA can obtain promising registration results, such as Stanford Buddha. However, the weight of each relative motion is estimated and assigned by some manual methods in wMA, it may assign a high weight to outliers, which can lead to the failure of multi-view registration.
Different from other competed methods, the proposed method utilizes the correntropy measure to achieve MA for muti-view registration. Compared with the Frobenius norm error, the correntropy measure can effectively alleviate the impact of large errors caused by outliers. For the balance of registration accuracy and convergence speed, adaptive kernel width has been selected by the well designed strategy. Therefore, the proposed method can achieve multi-view registration with promising results, even the input of relative motion set contains unreliable motions or outliers. The only weakness is that our method is less efficient than other competed methods due to weight calculation and a little more iterations.
In this paper, we proposed a novel and robust MA method for multi-view registration. To improve the robustness against outliers, it first utilizes the correntropy measure to design the objective function of MA, which arises a non-quadratic optimization problem. By the HQ theory, the correntropy based optimization problem can be solved by an alternating minimization procedure, which includes the operation of weight assignment and weighted MA derived from original MA algorithm. Further, the selection strategy of adaptive kernel width is proposed to balance the accuracy and convergent speed of our algorithm. Experiments tested on benchmark data sets illustrate that the proposed approach can achieve multi-view registration with better performance than existing state-of-the art methods on accuracy and robustness.
This work is supported by the Fundamental Research Funds Central Universities; in part by State Key Laboratory of Rail Transit Engineering Informatization (FSDI) under Grant Nos. SKLKZ19-01 and SKLK19-09. We also would like to thank Andrea Torsello for providing Angel and Hand datasets.
-  (2016) Global registration of 3d point sets via lrs decomposition. In European Conference on Computer Vision, pp. 489–504. Cited by: §1, §4.2.
-  (1992) Method for registration of 3-d shapes. In Sensor fusion IV: control paradigms and data structures, Vol. 1611, pp. 586–606. Cited by: §1.
-  (2016) Online variational bayesian motion averaging. In European Conference on Computer Vision, pp. 126–142. Cited by: §1.
Maximum correntropy kalman filter. Automatica 76 (1), pp. 70–77. Cited by: §1.
-  (2016) Generalized correntropy for robust adaptive filtering. IEEE Transactions on Signal Processing 64 (13), pp. 3376–3387. Cited by: §1.
-  (1992) Object modelling by registration of multiple range images. Image and vision computing 10 (3), pp. 145–155. Cited by: §1.
-  (2018) Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion. IEEE transactions on cybernetics 49 (4), pp. 1440–1453. Cited by: §1.
-  (2017) Joint alignment of multiple point sets with batch and incremental expectation-maximization. IEEE transactions on pattern analysis and machine intelligence 40 (6), pp. 1397–1410. Cited by: §1.
-  (2014) On averaging multiview relations for 3d scan registration. IEEE Transactions on Image Processing 23 (3), pp. 1289–1302. Cited by: §1, §4.2.
Lie-algebraic averaging for globally consistent motion estimation.
the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004., Vol. 1, pp. 1–8. Cited by: §1, §3.1.
-  (2006) Robustness in motion averaging. In Asian Conference on Computer Vision, pp. 457–466. Cited by: §1.
-  (2018) Weighted motion averaging for the registration of multi-view range scans. Multimedia Tools and Applications 77 (9), pp. 10651–10668. Cited by: §1, §4.2.
-  (2019) Robust matrix completion via maximum correntropy criterion and half-quadratic optimization. IEEE Transactions on Signal Processing 68, pp. 181–195. Cited by: §1.
-  (2018) Multi-view registration based on weighted lrs matrix decomposition of motions. IET Computer Vision 13 (4), pp. 376–384. Cited by: §1.
-  (2005) Global registration of multiple 3d point sets via optimization-on-a-manifold.. In Symposium on Geometry Processing, pp. 187–196. Cited by: §1.
-  (2) O: a general framework for graph optimization. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 3607–3613. Cited by: §1.
-  (2017) Fast descriptors and correspondence propagation for robust global point cloud registration. IEEE Transactions on Image Processing 26 (8), pp. 3614–3623. Cited by: §4.
-  (2014) Bayesian perspective for the registration of multiple 3d views. Computer Vision and Image Understanding 118, pp. 84–96. Cited by: §1.
-  (2007) The equivalence of half-quadratic minimization and the gradient linearization iteration. IEEE Transactions on Image Processing 16 (6), pp. 1623–1627. Cited by: §1, §2.2.
-  (2010) Information theoretic learning: renyi’s entropy and kernel perspectives. Springer. Cited by: §1.
-  (2001) Efficient variants of the icp algorithm.. In 3dim, Vol. 1, pp. 145–152. Cited by: §1.
-  (2011) Multiview registration via graph diffusion of dual quaternions. In CVPR 2011, pp. 2441–2448. Cited by: §4.
-  (1994) Zippered polygon meshes from range images. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 311–318. Cited by: §4.
Efficient registration of multi-view point sets by k-means clustering. Information Sciences 488, pp. 205–218. Cited by: §1.