1 Introduction
With the proliferation of threedimensional (3D) scanning devices, it is easy to acquire large volumes of 3D point clouds. Point clouds provide a direct and convenient way to describe outdoor and indoor scenes. However, the method is imposing when modeling such massive data. Automatic modeling methods typically produce excessively complex meshes, which are cumbersome for later applications. However, many applications (, simultaneous localization and mapping and levelofdetail generation) do not require overly fine models. Considering that manmade environments contain many planar structures, plane reconstruction remains a suitable choice for 3Dscene description. Furthermore, plane reconstruction or extraction is typically used as a prior step in various tasks, such as architecture modeling [2], place recognition [15], object recognition [30], and registration [42].
In manmade environments, planar structures generally conform to one of the following relationships: parallelism, orthogonality, coplanarity, and angular equality. Making full use of these relationships can considerably improve the robustness of the plane reconstruction algorithms for complex scenes [27]. However, traditional plane extraction techniques (, region growing, Hough transform [40, 4], and randomsample consensus (RANSAC) [16, 34]) do not take advantage of these geometric constraints and suffer from a lack of robustness under high percentages of noise and outliers.
Next, we refer to the task of reconstructing planes from 3D unstructured point clouds while maintaining a geometric relationship between reconstructed planes as regularityconstrained plane reconstruction (RCPR), which is increasingly used in industrial manufacturing and architectural modeling.
A trivial method of RCPR is the detectionthenregularization strategy, which has two steps. In the first step, the method extracts planes without regard to regularity constraints. Regularity is reconstructed in the second step. This method relies heavily on the quality of the extracted planes in the first step and is easily renders local solutions. Sophisticated algorithms, on the other hand, are traditionally timeconsuming, hampering practicality.
In this paper, we formulate RCPR as a global gradient minimization problem, proposing a simple but efficient algorithm to solve it. The proposed algorithm solves the problems of simultaneous plane reconstruction and regularization without losing efficiency. In fact, our algorithm is 31,000 times faster than extant RCPR algorithms.
2 Related work
In this section, we review works immediately related to our problem. We cover the following three main aspects: plane extraction, multimodel fitting, and regularityconstrained fitting.
Plane extraction
Plane extraction is an old but recurrent problem. It requires many different techniques and has many variants, depending on the tasks and input data. A very common technique is region growing, often employed by more sophisticated algorithms to extract initial planes [10, 27, 29, 22]. Another popular approach is RANSAC [16] and variants ( [34, 33, 18, 43]
). RANSAC is a robust data estimator when facing a high percentage of outliers. However, it was originally designed to cope with a single model and was not built to maintain regularity. The randomized Hough transform technique
[40] naturally handles multiple structures. However, it cannot cope with a high percentage of outliers [24].Recent planarstructure extraction techniques (, supervoxel segmentation [31, 23] and structuralscale planarshape detection [14]) constrain the scale of planar structures but do not constrain their geometric relationships.
In summary, traditional plane extraction methods do not aim to maintain regularity. However, they are often the initial step in the RCPR methods.
Multimodel fitting
The goal of multimodel fitting is to improve the robustness when fitting multiple models to data contaminated by noise and outliers. Multimodel fitting is typically formulated as an optimization problem. For example, Delong [12] and Amayo [1] converted the problem to a multilabeling problem and solved it via expansion [5] and a primaldual algorithm [9], respectively. Magri and Fusiello [25]
cast the multimodel fitting problem as a setcoverage problem and solved it with integer linear programming. Moreover, clusteringbased methods (Jlinkage
[37] and Tlinkage [24]) and hypergraphbased methods [39] have gained the popularity.In our application, because of the huge input point cloud, we favor methods having low time complexity. Unfortunately, most multimodel fitting methods do not meet this requirement.
Regularityconstrained fitting
Regularityconstrained fitting has gained popularity in recent years. Li [21] presented the GlobFit framework, which adopted an iterative detectionthenregularization strategy to fit primitives. It first used RANSAC to locally fit primitives and to detect the geometric relationships among the primitives. Then, it refitted these primitives using the detected constraints. Similarly, Zhou, and Neumann [41] adopted the same strategy to model buildings. Departing from GlobFit, Oesau [29] sought to progressively detect regularities rather than iterating between complete detection and regularization. Monszpart [27] focused on constraining the angles between pairs of planes. Their method first generated candidate planes with fitting distances. Then, it searched the optimal planes from candidates using mixedinteger linear programming. Although the method achieved good performance with cluttered scenes, it was timeconsuming because of the need to solve largescale mixedinteger programming problems. More recently, Joo [17] presented a fast branchandbound framework to estimate the optimal Manhattan frame.
3 Motivation and constraint models
RCPR’s regularity depends on the assumptions made of the real world. Here, we refer to this group of assumptions as a constraint model. Moreover, we define the constraint model as a collection of plane sets satisfying a particular constraint. Thus, the task of RCPR can be regarded as finding an element from the constraint model that best fits the given data. A typical constraint model is the Manhattan model [11].
Manhattan model
A plane set, , is a Manhattan model if and only if it satisfies
(1) 
where
is the set of unit normal vectors of
. Because a plane has two orientations, we specify that the angle between the normal vector in and positive zaxis is no more than .In practice, the Manhattan model is a very strong constraint. It can simplify a few tough problems, such as line clustering and vanishingpoint estimation in two dimensional (2D) images [3]. However, the real world usually does not conform to the Manhattan model. Therefore, Straub [36] proposed a mixture of Manhattan frames. The idea was to utilize multiple Manhattan frames (MMF) to represent complex scenes.
Multiple Manhattan frames model
A plane set , is an MMF model if it can be divided into nonoverlapping subsets, with each subset being a Manhattan model.
The limitation of Manhattan and MMF models is obvious. For example, they only consider parallelism and orthogonality. RAPTER [27] adopted a constraint model that considered more angular constraints. Because RAPTER’s constraint model is difficult to formulate, we only consider its variant, which we call the generalized Manhattan model.
Generalized Manhattan model
A plane set, , is a generalized Manhattan model, if and only if it satisfies
(2) 
where is a userdefined set of constraint angles. It is easy to note that, if , the generalized Manhattan model is equivalent to the Manhattan model.
A comparison of these constraint models on a linefitting task is illustrated in Fig. 1. We notice that the Manhattan model cannot fit this simple data well. The MMF obtains a better result but still misses one line. The generalized Manhattan model can fit the data well. However, it should introduce more angle constraints. For example, in Fig. 1, the set of constraint angles should be set to . Usually, it is difficult to know the exact constraint angles. Furthermore, more constraint angles considerably increase the complexity of regularization.
We consider, therefore, using implicit angle constraints instead of explicit angle constraints. Specifically, we introduce a parameter, , to constrain the number of different normal vectors of the reconstruction planes. Our motivation is based on a simple observation: In manmade scenes, the number of planes, , is always much more than the number of different normal vectors, . For example, in Fig. 1, we have eight lines, but only four different directions (i.e., normal vectors). We refer to our constraint model as a directional constraint (DC) model, defined as follows.
Directional constraint model
Given a plane set , we denote as the set of different normal vectors of . is a DC model, if and only if it satisfies
(3) 
Apart from the other three models, the DC model cannot maintain orthogonality. However, it gains the advantage of fewer priori parameters. In fact, the DC model only needs one priori parameter, , (i.e., the number of different normals). An example is given in Fig. 1 (d). There are eight lines but only four different directions. Therefore, we can set to obtain the correct fitting result. Moreover, by constraining the number of normal vectors, we implicitly establish the connection between planes, which helps obtain better fitting results under noise constraints and outliers (See Fig. 3).
In the next section, we discuss how to reconstruct a set of planes satisfying the DC model.
4 Methodology
4.1 Problem formulation
Given a point cloud, , equipped with normal vectors, , our task is to find a plane set,
, that best fits the input points while satisfying the DC model. If the input point cloud has no normal vector information, we use the principal component analysis algorithm to estimate their normal vectors.
Our method comprises two steps. First, we reconstruct the normal vector for each point while maintaining the constraints of DC model. Second, the desired planes can be easily obtained by grouping nearby points having the same reconstructed normal. Next, we focus on the first step, because the second step is easily implemented. More specifically, we consider the following problem:
(4) 
where, is the set of reconstructed normal vectors; is the energy that measures how well fits . Here, we adopt energy for , owing to its performance under high noise. Moreover, the constraint term, , can be relaxed by introducing a regularization parameter . Therefore, Eq. 4 is rewritten as
(5) 
where is the number of points; is the neighboring set of the th point; and is the index of for the th point. Note that Eq. 5 can also satisfy the DC model if we choose the appropriate value.
In Eq. 5, the first term, , is used to guide the output normal vectors as close as possible to the input. The second term, , is used to constrain the sparsity in the local sense. It guides the normal vector of each point close to its neighboring normals. The third term, , is used to constrain the global sparsity. Two parameters, and , are used to balance the influence of two constraint terms. Eq. 5 can be further divided into the following two subproblems:
Subproblem 1
(6) 
Subproblem 2
We notice that Eq. 6 is an gradient minimization problem and that Eq. 8 is a subset selection problem. This provides a simple notion: we can optimize Eq. 5 by alternately solving two subproblems. By reviewing the stateoftheart minimization [28] and subset selection techniques [13], we can provide a simple but efficient algorithm called, ”Global”, to solve the problem.
4.2 Global algorithm
The framework of the Global algorithm is given in Algorithm 1, comprising the following two steps.
minimization
Subset selection
To solve subproblem 8, we adopt greedy unconstrained submodular optimization. Let, . is a submodular function satisfying for any . Therefore, is an unconstrained submodular maximization (USM) problem. We can use the lineartime algorithm proposed in [8] to solve it.
However, the complexity of using Algorithm [8] directly is , which is unacceptable. We notice that, after performing regionfusionbased minimization, the normal vectors, , are divided into several disjoint regions. Each region shares the same normal vector. Therefore, can be simplified to
(9) 
where is the set of regions; is the normal vector of the th region; and is the set of normal vectors consisting of , .
Additionally, we can ignore the regions whose number of points is less than a threshold, . can be explained as the minimal support points for a plane (i.e., the filtered regions are treated as outliers). Thus, the time complexity of our subset selection step is reduced to . In practice, it can assume .
More details are described in Algorithm 2.
Algorithm 2 has a 1/3 approximation guarantee (i.e., the energy of its solution is always more than 1/3 times the optimal energy). The randomized version [8]
has a better approximate rate of 1/2. However, we choose the deterministic algorithm, because it is more stable. Furthermore, with a heuristic sequence (i.e., descending order by cardinality), our solution is usually better than the theoretical upper bound.
The Global algorithm can be seen as an extension of gradient minimization algorithms. The traditional gradient minimization methods only consider regularity between local neighbors. Our algorithm introduces global regularity by limiting the number of different normal vectors. It establishes implicit relationships between the disjoint regions. In Section 5, we demonstrate that these global relationships are very useful, especially in manmade environments.
Plane reconstruction
For a postprocessing, the constrained planes can be constructed from disjoint regions. Additionally, the normal vector of each plane is set to the reconstruction normal vector of the corresponding region.
4.3 Difference to the multilabeling problem
Eq. 5 looks like the multilabeling problem of [12, 1]. However, they are quite different. First, the energy formulation in [12, 1] only constrained the number of models (planes in our context). In contrast, Eq. 4 constrains the number of plane normals, because, it constrains the size of the label set. Second, in the multilabeling problems, the label set, (or in our formulation), is usually known. For example, Pearl [12] must first compute an initial set of models or labels via sampling. Our algorithm is designed to solve the problem without an initial label set. In other words, the size of the initial label set in our problem is much larger than the label size in multilabeling problems. Therefore, an efficient algorithm is desired.
5 Experiments
Our method was implemented in C++ and run on Linux Ubuntu 16.04 with one core of Intel i7 CPU (2.5 GHz) and 8GB memory.
5.1 Parameter setting
There are four parameters adopted in our method. They are , the minimal number of points that support a plane; , the number of neighbors for each point; , the local regularization for normal vectors in minimization; and , the global regularization for normal vectors. and are common parameters, and their settings have been fully studied. Here, we set and for the following experiments.
Fig. 2 demonstrates the effects of and . We use the same color to draw the points with the same normal vector. In Fig. 2(b–d), we first set and increase the value of . One can observe that, as the increases, the number of regions decreases. However, the normal vectors for each region are different (i.e., equals the number of regions). In Fig. 2(eh), we set and increase the value of . The number of regions and the different normal vectors decrease simultaneously. When reaches 10,000, only the three most frequently normal are preserved, satisfying the Manhattan model.
In practice, for data having high noise, we choose higher values for and . One can also simply set equal to the one in [28]. Additionally, if the user has a prior knowledge of the scene (the value of ), then one can choose an appropriate value.
5.2 2D line fitting
We first study the robustness of the proposed method on the 2D linefitting problem with respect to seqRANSAC, Jlinkage [37], Tlinkage [24], and Pearl [12]. We implement seqRANSAC, Jlinkage, and Tlinkage ourselves, whereas the implementation of Pearl is obtained from [38].
As shown in Fig. 3, we generate eight lines, four of which are parallel, and four of which form a square. Each line comprises of 100 inliers, contaminated by Gaussian noise and outliers of different percentages. When we say 50% outliers, it means that the number of outliers is half of the entire set. Note that the ground truth lines only have three different directions. Our method can benefit from this constraint.
The results are collected in Fig. 3, where is the rootmeansquare (RMS) error of the angles between the reconstruction of normal vectors and ground truth normals. It is defined as
(10) 
where represents the ground truth inliers.
As noise and outliers increase, SeqRANSAC, Jlinkage, Tlinkage, and Pearl miss the groundtruth line segments, and generate more false positives. Jlinkage and Tlinkage aggregate the preference set without exploiting the regularity of the data. Moreover, Jlinkage and Tlinkage rely on the selection of the initial set. The tendency of decreasing the percentage of inliers affects the performance of Jlinkage and Tlinkage. Pearl solves the linefitting problem by presenting an optimization framework. However, it only constrains the number of fitted lines and does not constrain the number of directions. It also relies on the selection of the initial set.
Our method achieves the best in all situations. The reason can be ascribed to the constraint on the number of directions and the independence of the initial set. Note that, the number of directions is a nonlocal constraint, which similar to nonlocal denoising methods [7]. When outliers reach 80% (20% inliers), our method generates two false positives and misses one line segment. However, the directions of the fitted lines are correct, and the number of different directions (including false positives) remains three. Moreover, our method has the best time performance, significantly faster than the other three methods.
5.3 Plane reconstruction on 3D manual point sets
We further study the robustness of the proposed method on 3D data with respect to RAPTER [27], piecewise planar surface reconstruction (PPSR) [10], and Pearl [12]. We obtain the implementation of RAPTER from [26]. Here, we use the regiongrowing technique of [10] to extract the initial planes for Pearl, leading to better reconstruction results.
Noisy dodecahedron
We first demonstrate the performance of these methods on a noisy 3D point set. As shown in Fig. 4, we sampled 100,000 points from a dodecahedron and perturbed each point by adding Gaussian noise, where is the diagonal length of the axisaligned bounding box of the original point cloud.
RAPTER is susceptible to Gaussian noise, and, because it relies on the initial planes obtained from region growing, its results will be less than satisfactory when region growing fails. PPSR improves the performance of traditional region growing by carefully selecting seed points. However, it still generates too many redundant planes. Pearl [12] obtains a much better result but loses accuracy at the boundaries. Benefiting from the robustness of optimization and parallel faces constraints, our method estimates the correct model. Furthermore, our method is much faster than the others.
Empire State Building with outliers
We next test our method with RAPTER on the Empire State Building with different level of outliers. The ground truth model is downloaded from [20]. We sample 1,000,000 points from groundtruth model and add 250,000 and 1,000,000 outliers, respectively.
In addition to , we evaluate the accuracy of the reconstruction planes by computing the RMS of the distance from the groundtruth points, , to the reconstruction planes, , i.e.,
(11) 
where, denotes the index of the plane closest to .
in Eq. 11 is used to penalize the insufficient reconstruction of planes. However, it does not penalize redundant planes, which is critical for point clouds with outliers. Therefore, for each inlier point, , we calculate its projection point, , on the reconstruction planes. Then, we evaluate the distance from to the nearest point using ground truth points, i.e.,
(12) 
Note that can penalize the inaccurate and redundant reconstruction of planes simultaneously.
As shown in the black boxes in Fig. 5, the normal vector distribution of the original point clouds is very noisy. Nevertheless, both methods achieved visually good plane reconstruction results. As a defect, neither method can produce a complete plane reconstruction at the top part of the building (see closeup views in the yellow boxes). The reason is that the small plane is more susceptible to outliers, resulting in incorrect reconstruction planes. Despite that, we can observe that RAPTER produces several planes with inaccurate directions; whereas our method produces relatively few but more accurate planes. In the closeup views in the red boxes, we also observe that our method outperforms RAPTER in details.
Method  #Planes  Time  

RAPTER 
195  3.5  1.1e4  6.5e5  1.3e5 
Ours 
171  1.8  1.5e4  3.8e5  55 
RAPTER 
211  3.5  1.1e4  7.8e5  1.5e5 
Ours  178  1.8  1.5e4  3.8e5  38 
Quantitative comparison results are summarized in Table 1. Compared to RAPTER, our method achieves better and but worse . The normal vectors reconstructed by our method are significantly more accurate than RAPTER. This is also indicated in the black boxes of Fig. 5. Furthermore, RAPTER is very time consuming, because it needs to solve a largescale mixedinteger programming system. In contrast, our method is approximately 2,000 times faster.
5.4 Plane reconstruction on realworld point clouds
To quantitatively evaluate the performance of the proposed method in practice, two realworld dataset are considered. In the following experiments, we set and . The parameters for sequential RANSAC and Pearl are finetuned.
We first test our method on the NYUdepth dataset [35]. This dataset was originally designed for object detection. We use it to quantitatively evaluate the performance of plane segmentation. A subset of 218 images, mainly containing planar objects, is selected from the dataset. The subset includes kitchen, bedroom, and bathroom scenes. Scenes containing too many nonplanar objects, such as offices and stores, are excluded, because our method is not designed for nonplanar scenes. Two standard segmentation metrics, global consistency error (GCE), and local consistency error (LCE) from [32], are introduced here for evaluation.
Method  GCE  LCE  Time 

seqRANSAC  0.44  0.36  7.98 
Pearl  0.36  0.28  60.11 
Gobal (Ours)  0.33  0.24  21.63 
The quantitative results are summarized in Table 2, where two typical scenes are shown in Fig. 6. It can be observed that our method is significantly superior to sequential RANSAC and Pearl in terms of visual and quantitative results.
We further test our method on indoor laserscanning point clouds ^{1}^{1}1https://www.ifi.uzh.ch/en/vmml/research/datasets.html, including five datasets: UZH IfI (12 data), UZH Irchel (10 data), ETH (18 data), Rooms detection datasets (9 data), and Full 3D (8 data). Owing to the memory limitations of PEARL, the input data was downsampled by performing Poissondisk sampling [6] using a resolution .
The quantitative comparison results are shown in Fig. 7 and some typical reconstruction results are shown in Fig. 8. Owing to the lack of ground truth, only and are evaluated here. Nevertheless, we can easily observe that our method is superior to the other two methods. PEARL has a high computational complexity and has difficulty maintaining smallscale planar structures. RANSACbased methods tend to merge unconnected planes, making it difficult to maintain the edges of local structures. The proposed method not only maintains the local structure, it also guarantees the global constraints. Therefore, we achieve the best results.
6 Conclusion
In this paper, we presented a fast algorithm for RCPR problems. Our algorithm was based on a constraint model, requiring less prior knowledge compared to traditional RCPR algorithms. We showed that, by considering the constraints, plane reconstruction results could significantly improved, especially for data with highlevel outliers and noise. Furthermore, our algorithm is efficient, between 3 and 1,000times faster than the existing RCPR algorithms.
References
 [1] P. Amayo, P. Piniés, L. M. Paz, and P. Newman. Geometric multimodel fitting with a convex relaxation algorithm. In Computer Vision and Pattern Recognition. IEEE, 2018.
 [2] M. Arikan, M. Wimmer, and S. Maierhofer. Osnap:optimizationbased snapping for modeling architecture. ACM Transactions on Graphics, 32(1):1–15, 2013.
 [3] J. C. Bazin, Y. Seo, C. Demonceaux, and P. Vasseur. Globally optimal line clustering and vanishing point estimation in Manhattan world. In Computer Vision and Pattern Recognition, pages 638–645. IEEE, 2012.
 [4] D. Borrmann, J. Elseberg, L. Kai, and A. Nüchter. The 3D Hough transform for plane detection in point clouds: A review and a new accumulator design. 3D Research, 2(2):32:1–32:13, 2011.
 [5] Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. In International Conference on Computer Vision, pages 377–384. IEEE, 2002.
 [6] R. Bridson. Fast poisson disk sampling in arbitrary dimensions. In ACM SIGGRAPH 2007 Sketches, SIGGRAPH ’07, New York, NY, USA, 2007. ACM.
 [7] A. Buades, B. Coll, and J. M. Morel. A nonlocal algorithm for image denoising. In Computer Vision and Pattern Recognition, pages 60–65. IEEE, 2005.
 [8] N. Buchbinder, M. Feldman, J. Naor, and R. Schwartz. A tight linear time (1/2)approximation for unconstrained submodular maximization. In Foundations of Computer Science, pages 649–658, 2012.
 [9] A. Chambolle and T. Pock. A firstorder primaldual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging & Vision, 40(1):120–145, 2011.
 [10] A. L. Chauve, P. Labatut, and J. P. Pons. Robust piecewiseplanar 3d reconstruction and completion from largescale unstructured point data. In Computer Vision and Pattern Recognition, pages 1261–1268. IEEE, 2010.

[11]
J. M. Coughlan and A. L. Yuille.
Manhattan world: compass direction from a single image by Bayesian inference.
In International Conference on Computer Vision, volume 2, pages 941–947. IEEE, 1999.  [12] A. Delong, A. Osokin, H. N. Isack, and Y. Boykov. Fast approximate energy minimization with label costs. International Journal of Computer Vision, 96(1):1–27, 2012.
 [13] E. Elhamifar and M. C. D. P. Kaluza. Online summarization via submodular and convex optimization. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1818–1826, 2017.
 [14] H. Fang, F. Lafarge, and M. Desbrun. Planar shape detection at structural scales. In Computer Vision and Pattern Recognition, Salt Lake City, United States, 2018. IEEE.
 [15] E. FernandezMoral, W. MayolCuevas, V. Arevalo, and J. GonzalezJimenez. Fast place recognition with planebased maps. In International Conference on Robotics and Automation, pages 2719–2724. IEEE, 2013.
 [16] M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):726–740, 1981.
 [17] K. Joo, T. Oh, J. Kim, and I. S. Kweon. Robust and globally optimal Manhattan frame estimation in near real time. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–1, 2018.
 [18] S. Korman and L. Roee. Latent RANSAC. In Computer Vision and Pattern Recognition, pages 6693–6702. IEEE, 2018.
 [19] F. Lafarge and P. Alliez. Surface reconstruction through point set structuring. Computer Graphics Forum, 32(2):225–234, 2013.
 [20] F. Lafarge and P. Alliez. Surface reconstruction through point set structuring. http://wwwsop.inria.fr/members/Florent.Lafarge/benchmark/surface_reconstruction/reconstruction.html, 2013.
 [21] Y. Li, X. Wu, Y. Chrysathou, A. Sharf, D. CohenOr, and N. J. Mitra. Globfit: Consistently fitting primitives by discovering global relations. ACM Transaction on Graphics, 30(4):52:1–52:12, 2011.
 [22] Y. Lin, C. Wang, B. Chen, D. Zai, and J. Li. Facet segmentationbased line segment extraction for largescale point clouds. IEEE Transactions on Geoscience & Remote Sensing, 55(9):4839–4854, 2017.
 [23] Y. Lin, C. Wang, D. Zhai, W. Li, and J. Li. Toward better boundary preserved supervoxel segmentation for 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 143:39–47, 2018.
 [24] L. Magri and A. Fusiello. Tlinkage: A continuous relaxation of JLinkage for multimodel fitting. In Computer Vision and Pattern Recognition, pages 3954–3961. IEEE, 2014.
 [25] L. Magri and A. Fusiello. Multiple models fitting as a set coverage problem. In Computer Vision and Pattern Recognition, pages 3318–3326. IEEE, 2016.
 [26] A. Monszpart, N. Mellado, G. J. Brostow, and N. J. Mitra. RAPter implementation. http://geometry.cs.ucl.ac.uk/projects/2015/regulararrangementsofplanes/, 2015.
 [27] A. Monszpart, N. Mellado, G. J. Brostow, and N. J. Mitra. RAPter: rebuilding manmade scenes with regular arrangements of planes. Acm Transactions on Graphics, 34(4):103, 2015.
 [28] R. M. H. Nguyen and M. S. Brown. Fast and effective gradient minimization by region fusion. In IEEE International Conference on Computer Vision, pages 208–216, 2015.
 [29] S. Oesau, F. Lafarge, and P. Alliez. Planar shape detection and regularization in tandem. Computer Graphics Forum, page 14, 2015.
 [30] S. Oesau, F. Lafarge, and P. Alliez. Object classification via planar abstraction. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, III3:225–231, 2016.
 [31] J. Papon, A. Abramov, M. Schoeler, and F. Wörgötter. Voxel cloud connectivity segmentation  supervoxels for point clouds. In Computer Vision and Pattern Recognition, pages 2027–2034. IEEE, 2013.

[32]
M. Polak, H. Zhang, and M. Pi.
An evaluation metric for image segmentation of multiple objects.
Image and Vision Computing, 27(8):1223–1227, 2009.  [33] R. Raguram, O. Chum, M. Pollefeys, J. Matas, and J. M. Frahm. USAC: a universal framework for random sample consensus. IEEE Transactions on Pattern Analysis & Machine Intelligence, 35(8):2022–2038, 2013.
 [34] R. Schnabel, R. Wahl, and R. Klein. Efficient RANSAC for point‐cloud shape detection. Computer Graphics Forum, 26(2):214–226, 2010.
 [35] N. Silberman, D. Hoiem, P. Kohli, and R. Fergus. Indoor segmentation and support inference from RGBD images. In European Conference on Computer Vision, pages 746–760, 2012.
 [36] J. Straub, G. Rosman, O. Freifeld, J. J. Leonard, and J. W. Fisher. A mixture of Manhattan frames: Beyond the Manhattan world. In Computer Vision and Pattern Recognition, pages 3770–3777. IEEE, 2014.
 [37] R. Toldo and A. Fusiello. Robust multiple structures estimation with JLinkage. In European Conference on Computer Vision, pages 537–547. IEEE, 2008.
 [38] O. Veksler and A. Delong. Pearl implementation. http://vision.csd.uwo.ca/code/, 2012.
 [39] H. Wang, G. Xiao, Y. Yan, and D. Suter. Searching for representative modes on hypergraphs for robust geometric model fitting. IEEE Transactions on Pattern Analysis & Machine Intelligence, PP(99):1–1, 2018.
 [40] L. Xu, E. Oja, and P. Kultanen. A new curve detection method: Randomized Hough transform (RHT). Pattern Recognition Letters, 11(5):331–338, 1990.
 [41] Q. Y. Zhou and U. Neumann. 2.5D building modeling by discovering global regularities. In Computer Vision and Pattern Recognition, pages 326–333. IEEE, 2012.
 [42] Z. Zhou, H. Jin, and Y. Ma. Robust planebased structure from motion. In Computer Vision and Pattern Recognition, pages 1482–1489. IEEE, 2012.
 [43] M. Zuliani, C. S. Kenney, and B. S. Manjunath. The multiRANSAC algorithm and its application to detect planar homographies. In International Conference on Image Processing, volume 3, pages III–153. IEEE, 2005.