1 Introduction
Point set registration is a fundamental and important technique in many domains, such as compute vision, robotics, and computer graphics, etc. For each range scan given in a setcentered frame, the registration goal is to find an optimal rigid transformation (global motion) and transform it into the reference coordinate frame. Due to the number of involved point sets, point set registration can be divided into pairwise registration and multiview registration problems. In the past few decades, lots of effective approaches have been proposed to solve the pairwise registration problem. Among these approaches, the iterative closest point (ICP) algorithm
[2] is one of the most popular methods. Based on this basic algorithm, many ICP variants [21] have been proposed to improve the performance of pairwise registration in different perspectives. For convenience, we will use the term rigid transformation and motion interchangeable throughout this paper.Different from pairwise registration, the multiview registration problem is more complex and has attracted less attention. In the literature, some approaches have been proposed to solve this difficult problem. For example, Chen et al. [6] proposed the alignmentandmerging approach, which repeatedly aligns and merges two scans until all scans are merged into the whole model. This approach is straightforward but suffers from the error accumulation problem. To address this issue, Evangelidis et al. [8]
proposed the JRMPC approach, which assumes that all points are realizations of a unique Gaussian mixture model (GMM) and therefore casts the registration into clustering problem. Subsequently, the expectation maximization (EM) algorithm is utilized to estimate GMM parameters as well as all global motions for multiview registration. This approach is timeconsuming due the large number of parameters required to be estimated. Therefore, Zhu et al.
[24] introduced the means algorithm to solve the multiview registration problem. Compared with the JRMPC, the means based approach is more efficient and likely to obtain better registration results.As these approaches estimate each global motion sequentially, they are more likely to be trapped into a local minimum, especially when the scan number is large. Therefore, Krishnan et al. [15] proposed the optimizationonamanifold approach to simultaneously optimize all motions. To obtain desired results, it requires to establish accurate point correspondences for all scan pairs, which is very difficult in practice. Subsequently, Mateo et al. [18] extended this approach under the Bayesian perspective, which views pairwise correspondence as missing data and solves the registration problem by the EM algorithm. Although this approach can simultaneously optimize all global motions, it requires to compute a huge number of latent variables, which is time consuming.
For multiview registration, another feasible solution is to recover global motions from a set of relative motions. To this end, Govindu [10] proposed the motion averaging (MA) algorithm, which avoids averaging of motion in Lie groups but performs average in the Liealgebra of the underlying motion representation. With an initial guess, global motions can be simultaneously recovered from a set of relative motions by MA algorithm, which was further extent to solve multiview registration problem [9]. Although these two algorithm is effective, it is sensitive to outliers due to utilizing Frobenius norm error in optimization. Govindu [11] combined graphbased sampling scheme and Random sample consensus (RANSAC) method to remove motion outliers. This approach is more robust, but the efficiency is seriously reduced with the increase of scan number. For robot mapping, Grisetti et al. [16] proposed the general framework for graph optimization, called as G2O, which takes the same inputs as that of MA algorithm. Similar to MA algorithm, it is effective but sensitive to outliers.
Besides, Bourmaud et al. [3] proposed Bayesian MA algorithm for robot mapping. It is more complex than the original MA and its performance is greatly affected by the assignment of a reasonable covariance to each relative motion, which is very difficult in real applications. Meanwhile, Arrigoni et al. [1] introduced the lowrank and sparse (LRS) matrix decomposition to solve multiview registration, which concatenates all available relative motions into a large matrix and then decomposes it into one sparse matrix and one lowrank matrix. This approach can be viewed as another MA method and that is robust to outliers, but it requires more relative motions to achieve good registration. What’s more, these methods treat each relative motion equally, and this will reduce the performance of registration. Accordingly, Guo et al. [12] proposed weighted MA algorithm and Jin et al. [14] proposed weighted LRS algorithm, which can really improve the performance of multiview registration with each relative motion assigned by a suitable weight, e.g. reliable motions assigned with high weights. However, it is difficult to manually assign a suitable weight to each relative motion.
Previous MA methods often use Frobenius norm error in optimization, and they perform well under the assumption of Gaussian noises. However, in practice, a relative motion set often includes outliers. In this case, the Frobenius norm error can not properly capture error statistics, which may seriously degrade the performance. Recently, correntropy [20] has been proposed as an information theoretic learning measure to solve robust learning problems [4, 5, 7, 13]
. Compared with Frobenius norm, correntropy includes all even moments of the error. Therefore, the correntropy measure is robust against outliers and can achieve better learning performance especially when data contain large outliers.
Accordingly, this paper introduces the correntropy measure to reformulate the MA problem, which is difficult to be solved directly. To this end, the halfquadratic (HQ) [19] technique is utilized to transform the problem into a halfquadratic optimization problem, which can be solved by the traditional optimization method. Further, we design an adaptive selection strategy for kernel width to take advantage of correntropy properties. Compared with Frobenius norm error, the negative effects of outliers are therefore alleviated by the correntropy measure. In summary, the main contributions of this paper are delivered as 1) It proposes a novel cost function for robust motion averaging. 2) It develops an effective MA algorithm by the HQ technique. 3) Experiments carried out on benchmark data sets confirm its superior performance over other stateoftheart algorithms.
The remainder of the paper is organized as follows. Section 2 briefly briefly reviews the concepts of MCC and HQ optimization theory. Section 3 formulates the correntropy based objective function for motion averaging and proposes the HQ based algorithm. Following that is section 4, in which the proposed approach is tested and evaluated on four benchmark data sets. Finally, conclusions are drawn in Section 5.
2 Preliminaries
This section briefly reviews MCC and HQ optimization theory, which are bases of the proposed approach.
2.1 Maximum correntropy criterion
Given two random variables
and , the correntropy is defined by:(1) 
where denotes a shiftinvariant Mercer kernel and
is the joint probability distribution function (PDF) of
. In practice, the joint PDF is unknown and only a finite number of data points are available. With finite samples , the correntropy can be approximated as:(2) 
Usually, the correntropy kernel utilizes Gaussian Kernel:
(3) 
where is the kernel width and is the error term.
Obviously, the correntropy is a local and nonlinear similarity measure between two random variables within a ”window” in the joint space defined by the kernel width. Compared with traditional measures, the correntropy contains all the even moments of the difference between and
, and it is robust to outliers. In supervised learning, the correntropy measure based loss function is usually given by:
(4) 
which is referred to as the MCC.
2.2 Halfquadratic optimization theory
Usually, it is difficult to directly optimize the correntropy based objective function, which is nonquadratic. Therefore, the HQ technique has been introduced to solve this problem.
According to the HQ theory [19], there is a convex conjugated function corresponding to and they have the following relationship:
(5) 
where and the maximum is achieved at . Equivalently, Eq. (5) can also be transformed into:
(6) 
By defining and , Eq. (6) can be further derived as:
(7) 
Based on the HQ technique, the nonquadratic cost function is reformulated as the augmented objective function in enlarged parameter space by introducing auxiliary variable .
3 Robust Motion Averaging under MCC
This section states the MA problem in multiview registration and then proposes a robust solution under MCC.
3.1 Problem statement
Given multiple rang scans, the goal of multiview registration is to estimate the rigid transformation for each scan to the reference coordinate frame. For simplicity, the rigid transformation can be defined in the form of motion as:
(8) 
where and
denote the rotation matrix and translate vector, respectively. Compared with the multiview registration problem, the pairwise registration problem is much easier. Therefore, it is reasonable to achieve multiview registration based on pairwise registration, which arises the MA problem. Given a set of estimated relative motions
, it requires to recover the global motion for multiview registration. Accordingly, the multiview registration can be formulated the following optimization problem:(9) 
As either or denotes the variable of global motion, we only preserve as the variable for the simplicity. This problem has been solved by the original MA algorithm [10], which is sensitive to outliers due to the application of Frobenius norm error in the optimization.
To improve the robustness, we introduce correntropy as the error measure and reformulate the multiview registration problem as the following optimization problem:
(10) 
Eq. (10) denotes a nonconvex and nonquadratic cost function, which is difficult to be directly minimized by traditional methods. To this end, the HQ technique should be utilized to minimize this function.
3.2 Optimization by the HQ theory
As shown in Eq. (7), minimizing the correntropy measure based loss function in terms of equals to minimizing an augmented cost function in an enlarged parameter space . Accordingly, the correntropy measure based objective function can be further formulated as:
(11) 
Further, we can define the augmented cost function:
(12) 
According to the HQ optimization theory, we obtain the equivalent relation as follows:
(13) 
This optimization problem can then be solved by the alternating minimization procedure as follows:
(1) Optimization of : According to Eq. (5) and Eq. (7), the minimum of the objective function is achieved by for given a certain . Therefore, the optimal solution of can be estimated for the fixed as:
(14) 
This procedure can be viewed as the weight assignment operation, which assigns different weights to each relative motion based on the residual motion error. According to the property of Gaussian function, a relative motion with small error will be assigned with a large weight, and vice versa. Different from previous methods, we do not manually estimate a weight for each relative motion, but automatically calculate them by the residual motion error. Therefore, suitable weight can be assigned to each relative motion due to properties of the correntropy measure.
(2) Optimization of : For the fixed , Eq. (12) is simplified into the following optimization problem:
(15) 
Eq. (15) denotes the weighted MA problem, where the negative impact of outliers can be seriously reduced due to the small weight assigned by the first procedure. Since this problem can be solved by the variant of original MA algorithm, we present the solution without any provement.
3.3 Weighted motion averaging
Given the relative motion set , the motion averaging algorithm requires initial global motions to achieve multiview registration by iterations. For one relative motion and previous global motion , the residual relative motion is defined as:
(16) 
Eq. (16) can be converted into the equivalent formulation:
(17) 
where . Subsequently, the function is utilized to extract parameters from to form a column wise vector and then Eq. (17) is transformed into the following form:
(18) 
where .
As each relative motion is assigned with a weight in our approach, the blockmatrix is constructed with the th and th blockelements filling with and :
(19) 
where denotes the identity matrix. According to Eq. (18), there exists the following relationship:
(20) 
where . To refine global motions, Eq. (20) can be extended to the situation of many relative motions as follows:
(21) 
where and . This formulation leads to vector including parameters of all residual global motions:
(22) 
where is the pseudoinverse matrix of . Finally, elements of can be utilized to update each global motion as:
(23) 
where is residual global motions and denotes the inverse function of .
3.4 Implementation
Obviously, our method is local convergent. To obtain desired results, initial guess should be provided for global motions in advance. Besides, its performance is affected by the kernel width in correntropy measure. In the literature, lots of works have illustrated that relatively large can offer high convergence speed but suffer from less accuracy, and vice versa. As our approach achieves multiview registration by iterations, it is better to use adaptive kernel width. Specifically, the kernel width is set to be large at the beginning of the iteration and it should decrease with the increase of the iteration number. As residual motion error of decreases with the increase of iteration number, it is reasonable to set the kernel width to be proportional with the residual error of all global motions, e.g. , where is a preset parameter and denotes the residual motion error defined as:
(24) 
This setting can well balance the convergence speed and accuracy of the proposed method.
Based on the above description, the proposed method is summarized in Algorithm 1, where the parameter will be discussed and determined in experiments.
4 Experiments
Dataset  Armadillo  Buddha  Bunny  Dragon  Hand 

Scan  12  15  10  15  36 
Point  307625  469193  362272  1099005  1605575 
Motion  68  79  46  103  323 
Translation error  

Mean  Median  RMSE  Mean  Median  RMSE  
Armadillo  0.0059  0.0041  0.0054  0.7365  0.4666  0.7052 
Buddha  0.1984  0.0124  0.6150  1.8631  0.7141  4.4277 
Bunny  0.0357  0.0076  0.0880  0.0906  2.4320  6.1294 
Dragon  0.1368  0.0061  0.4930  4.5063  0.6799  12.9122 
Hand  0.0103  0.0026  0.0625  0.8366  0.2411  4.2900 
This section tests and evaluates our approach on five benchmark data sets, where four data sets are taken from the Stanford 3D Scanning Repository [23] and the Hand data set is provided by Torsello [22]. Each of them was acquired from one object model in multiple views and ground truth of rigid transformations was provided with multiple scans for the evaluation of registration results. But they are only utilized to assist for the final assessment. All experiments are performed on a fourcore 3.6 GHz computer with 8 GB of memory.
As the proposed approach takes relative motions as its input to recover global motions, we estimate relative motions for each scan pair by utilizing the pairwise registration method proposed in [17], which can obtain reliable results for these scan pairs with nonlow overlap percentage. Given a set of scans, many scan pairs contain low overlap or nonoverlapping percentages, their estimated relative motions are unreliable and meaningless. For accurate registration, it is better to utilize as many reliable relative motions but few unreliable relative motions as possible. Therefore, we only select relative motions of these scan pairs, whose trimmed mean square errors are less than the predefined threshold. For accuracy comparison, the registration error of rotation matrix and translation vector are defined as and , respectively. Here, indicates the ground truth of the th rigid transformation and denotes the one estimated by multiview registration approach. Table I and Fig. 1 demonstrate some details of these four data sets as well as preserved motion sets. Besides, Table II we list statistics information of all relative motions in each data set, including the mean, median, RMSE of rotation and translation errors. As shown in Table II, each preserved motion set still contains outliers.
4.1 Parameter tuning
The performance of our approach is related to the selection of kernel width , which is set to be . Empirically, we can set , which directly assigns the residual motion error to the kernel width. Here, we do experiments to find its appropriate value and check whether the performance of our method is sensitive to this parameter. More specially, we change the value of in our approach and test it on all four data sets. Experimental results are reported in the form of registration errors . During the experiment, we find the setting of around 1.0 is more likely to obtain promising results for multiview registration. Fig. 4 records registration results of our approach with varied values of around 1.0.
As shown in Fig. 4, we can observed that: 1) the setting of tends to obtain the desired results. 2) The performance of our approach is stable as long as is set to be within a certain value range, e.g. . Accordingly, the proposed approach is robust to the parameter as long as it is chosen from a reasonable range, which makes it easy to apply this approach without much effort for parameter tuning. However, both too large and too small may result in undesired registration results. For large , the correntropy measure is difficult to discriminate outliers from all relative motions, so the proposed method is unable to recover accurate global motions from a set of relative motions including outliers. For small , inliers with small noises may be viewed as outliers, which also makes the proposed method be unable to obtain desired registration results. Considering all these factors, we set , i.e. in the proposed method in the following experiments.
Buddha  Bunny  Dragon  Hand  

T(s)  Time(s)  T(s)  T(s)  T(s)  
LRS  0.3223  8.8161  0.0508  0.1985  1.8254  0.0814  0.0649  3.5322  0.0353  0.2760  12.0721  0.0722  0.0187  1.3681  0.5960 
G2O  0.3223  8.8161  0.6108  0.1985  1.8254  0.8414  0.0649  3.5322  0.5004  0.2760  12.0721  0.6895  0.0142  0.8677  2.5293 
MA  0.6448  10.0962  0.6842  0.4554  3.4936  1.1046  0.0738  4.5743  0.1276  0.8135  20.5170  0.9533  0.0196  1.6076  9.3326 
wMA  0.1216  2.9548  0.5562  0.0143  1.2425  0.8419  0.0380  2.2822  0.1438  0.2583  13.7669  0.4624  0.0345  2.6428  9.8012 
Ours  0.0061  0.4393  0.6523n  0.0103  0.8571  1.2072  0.0121  0.6740  0.2784  0.0213  1.5517  1.0528  0.0078  0.4967  15.7183 
4.2 Results
To demonstrate the performance, the proposed method is tested on four data sets and compared with some related approaches, including the multiview registration approach based on the lowrank and sparse decomposition algorithm [1], original motion averaging algorithm [9], and weighted motion averaging algorithm [12], which are abbreviated as LRS, WA, and wWA, respectively. It should be noted that LRS does not requires initial guess for multiview registration, but all other three approaches require initial global motions. As only relative motions are available in each data set, the output of LRS is taken as the input of other three approaches for multiview registration. Experimental results are reported in the form of run time, rotation error, and translation error. These registration results are all recorded in Table III. For the evaluation of registration accuracy in a more intuitive manner, Fig. 3 displays all multiview registration results in the form of a crosssection.
As shown in Table III and Fig. 3, LRS is robust to unreliable relative motions. Without the initial guess of global motions, LRS may efficiently achieve multiview registration by utilizing a set of relative motions. However, it requires a high proportion of available relative motions to obtain promising registration results. As shown in Fig. 1, proportions of available relative motions are all below for these four data sets, so it is difficult to obtain promising registration results due to a low proportion of available relative motions.
Similar to robot mapping, the multiview registration can be achieved by the G2O method, which takes relative motions and some results of LRS as its inputs. Besides, each relative motion requires to be assigned with one covariance matrix to denotes its uncertainty or reliability. Here, we assign the identity matrix to each relative motion due to the lack of prior information. As Table III and Fig. 3 demonstrate, the G2O method is efficient but is unable to obtain promising registration results. To obtain the desired results, each relative motion requires one appropriate covariance matrix, which is very difficult in most of practical applications.
For multiview registration, both MA and wMA take registration results of LRS as their initial guess. As MA utilizes Frobenius norm error for the estimation of global motions, it is sensitive to outliers and difficult to obtain promising registration results due to the exiting of outliers. Different from MA, wMA pays more attention to reliable relative motions by assigning high weights. When each relative motion is assigned with one appropriate weight, e.g. outliers assigned with very low weight, wMA can obtain promising registration results, such as Stanford Buddha. However, the weight of each relative motion is estimated and assigned by some manual methods in wMA, it may assign a high weight to outliers, which can lead to the failure of multiview registration.
Different from other competed methods, the proposed method utilizes the correntropy measure to achieve MA for mutiview registration. Compared with the Frobenius norm error, the correntropy measure can effectively alleviate the impact of large errors caused by outliers. For the balance of registration accuracy and convergence speed, adaptive kernel width has been selected by the well designed strategy. Therefore, the proposed method can achieve multiview registration with promising results, even the input of relative motion set contains unreliable motions or outliers. The only weakness is that our method is less efficient than other competed methods due to weight calculation and a little more iterations.
5 Conclusions
In this paper, we proposed a novel and robust MA method for multiview registration. To improve the robustness against outliers, it first utilizes the correntropy measure to design the objective function of MA, which arises a nonquadratic optimization problem. By the HQ theory, the correntropy based optimization problem can be solved by an alternating minimization procedure, which includes the operation of weight assignment and weighted MA derived from original MA algorithm. Further, the selection strategy of adaptive kernel width is proposed to balance the accuracy and convergent speed of our algorithm. Experiments tested on benchmark data sets illustrate that the proposed approach can achieve multiview registration with better performance than existing stateofthe art methods on accuracy and robustness.
Acknowledgements
This work is supported by the Fundamental Research Funds Central Universities; in part by State Key Laboratory of Rail Transit Engineering Informatization (FSDI) under Grant Nos. SKLKZ1901 and SKLK1909. We also would like to thank Andrea Torsello for providing Angel and Hand datasets.
References
 [1] (2016) Global registration of 3d point sets via lrs decomposition. In European Conference on Computer Vision, pp. 489–504. Cited by: §1, §4.2.
 [2] (1992) Method for registration of 3d shapes. In Sensor fusion IV: control paradigms and data structures, Vol. 1611, pp. 586–606. Cited by: §1.
 [3] (2016) Online variational bayesian motion averaging. In European Conference on Computer Vision, pp. 126–142. Cited by: §1.

[4]
(2017)
Maximum correntropy kalman filter
. Automatica 76 (1), pp. 70–77. Cited by: §1.  [5] (2016) Generalized correntropy for robust adaptive filtering. IEEE Transactions on Signal Processing 64 (13), pp. 3376–3387. Cited by: §1.
 [6] (1992) Object modelling by registration of multiple range images. Image and vision computing 10 (3), pp. 145–155. Cited by: §1.
 [7] (2018) Robust graphbased semisupervised learning for noisy labeled data via maximum correntropy criterion. IEEE transactions on cybernetics 49 (4), pp. 1440–1453. Cited by: §1.
 [8] (2017) Joint alignment of multiple point sets with batch and incremental expectationmaximization. IEEE transactions on pattern analysis and machine intelligence 40 (6), pp. 1397–1410. Cited by: §1.
 [9] (2014) On averaging multiview relations for 3d scan registration. IEEE Transactions on Image Processing 23 (3), pp. 1289–1302. Cited by: §1, §4.2.

[10]
(2004)
Liealgebraic averaging for globally consistent motion estimation.
In
the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004.
, Vol. 1, pp. 1–8. Cited by: §1, §3.1.  [11] (2006) Robustness in motion averaging. In Asian Conference on Computer Vision, pp. 457–466. Cited by: §1.
 [12] (2018) Weighted motion averaging for the registration of multiview range scans. Multimedia Tools and Applications 77 (9), pp. 10651–10668. Cited by: §1, §4.2.
 [13] (2019) Robust matrix completion via maximum correntropy criterion and halfquadratic optimization. IEEE Transactions on Signal Processing 68, pp. 181–195. Cited by: §1.
 [14] (2018) Multiview registration based on weighted lrs matrix decomposition of motions. IET Computer Vision 13 (4), pp. 376–384. Cited by: §1.
 [15] (2005) Global registration of multiple 3d point sets via optimizationonamanifold.. In Symposium on Geometry Processing, pp. 187–196. Cited by: §1.
 [16] (2) O: a general framework for graph optimization. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 3607–3613. Cited by: §1.
 [17] (2017) Fast descriptors and correspondence propagation for robust global point cloud registration. IEEE Transactions on Image Processing 26 (8), pp. 3614–3623. Cited by: §4.
 [18] (2014) Bayesian perspective for the registration of multiple 3d views. Computer Vision and Image Understanding 118, pp. 84–96. Cited by: §1.
 [19] (2007) The equivalence of halfquadratic minimization and the gradient linearization iteration. IEEE Transactions on Image Processing 16 (6), pp. 1623–1627. Cited by: §1, §2.2.
 [20] (2010) Information theoretic learning: renyi’s entropy and kernel perspectives. Springer. Cited by: §1.
 [21] (2001) Efficient variants of the icp algorithm.. In 3dim, Vol. 1, pp. 145–152. Cited by: §1.
 [22] (2011) Multiview registration via graph diffusion of dual quaternions. In CVPR 2011, pp. 2441–2448. Cited by: §4.
 [23] (1994) Zippered polygon meshes from range images. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 311–318. Cited by: §4.

[24]
(2019)
Efficient registration of multiview point sets by kmeans clustering
. Information Sciences 488, pp. 205–218. Cited by: §1.