For many applications in computer graphics and related domains, surface normals are an important property of 3D point cloud or mesh data. While normal directions
, i.e. the sign of the normal vectors, generally is a much harder problem. In particular for mesh data, there exists a wide variety of approaches based on principles such as voting, visibility, propagation, and optimization. Since point clouds are increasingly used as a means for representing various kinds of objects and scenes in fields like architecture, design, archaeology, and cultural heritage, methods working directly on point clouds have also received attention. A particularly important and discriminating aspect of any method is the severity of the assumptions made on the input and the particular geometry represented by the data. Regarding the input data, the assumptions range from very restrictive such as watertight, connected meshes, to unconnected polygon soups, possibly with missing parts. Point clouds pose additional challenges for certain kinds of methods based on connectivity for propagation, or surfaces for performing ray casting against, since such information is not directly available from the data. With respect to the class of the underlying object or scene, some methods make the assumption that the object itself is a closed 2-manifold. While this assumption simplifies the task of distinguishing between inside and outside space, many kinds of larger-scale datasets such as 3D urban environments do not fulfill this requirement.
Our work targets the challenging task of automatically determining normal orientations in completely unstructured 3D point cloud datasets of building interiors with multiple stories and rooms. We are specifically interested in the main structure of the building consisting of floor, ceiling, and wall surfaces. This information is an important prerequisite for e.g. reconstruction tasks aiming at automatic generation of higher-level 3D models from point cloud data. Clearly, knowledge about correct surface orientation in previously unstructured data greatly helps to distinguish between room interior, interior of wall volumes, and outside area. Point cloud scans of building interiors, possibly including parts of exterior façade and parts of the outside area scanned through windows, pose two particular challenges. First, such datasets usually consist of millions of points and cover a relatively large area which requires efficient means of processing them. Second, the constellation of rooms within a building can be quite complex, yielding a much more intricate surface topology than 2-manifolds.
The proposed method for automatically orienting normals in point clouds of building interiors combines different ideas to provide efficient processing of real-world scans. We first simplify the scene by detecting planes in the point cloud and subsequently working on surface patches instead of individual points. One advantage of working on patches instead of individual points is the drastically reduced computational complexity. In addition, the surface representation enables us to employ a specifically tailored path tracing approach to estimate which side of each patch is probably room interior, wall, or outside area. While visibility information is exploited by several algorithms, our method not only takes direct visibility into account but also higher-order visibility through multiple ray bounces. Using this initial, per-patch estimation, we then vote for a global orientation for each surface to increase the robustness of the estimation. Finally, the determined surface orientations are used to flip the normal orientations of the points belonging to the respective surface. Our approach is evaluated on multiple real-world datasets for which ground truth normal orientations for comparison are available by means of known scanner positions.
In summary, the contribution of our approach is a fast and fully automatic normal orientation estimation for the challenging scenario of indoor building scans without strong assumptions on the input data. The results of our method can greatly facilitate tasks such as reconstruction of 3D models from point clouds which require knowledge about the orientation of surfaces of the main building structure such as floors, ceilings, and walls.
2 Related Work
A classical propagation-based approach for orienting normals of point sets is described by Hoppe et al. . It derives a consistent orientation of tangent planes for data points by means of solving an optimization problem on the Riemannian graph of the points with edge weights proportional to the normal direction deviation between neighboring points. The method is only applicable for densely sampled, closed surfaces and may fail at sharp creases. König et al.  base their method on the method by Hoppe et al.  but propose a new unreliability cost for traversing the Riemannian graph based on Hermite curves. One recent point cloud based approach by Schertler et al.  generalizes propagation as a graph-based energy minimization problem. To this end, the graph-based idea by Hoppe et al.  is reformulated to a maximum-likelihood problem on a Markov random field. They also propose to use the streaming approach by Pajarola  to perform out-of-core processing.
The volumetric approach to solid inside/outside classification of polygonal data by Murali et al.  is based on a partitioning of space into polyhedral cells on which a consistent classification is derived by optimization. Xie et al.  segment an input point cloud into so-called mono-oriented regions through an active contour method. Subsequently, a consistent inside/outside partitioning is achieved by means of a voting algorithm. The approach by Mello et al.  constructs an adaptively subdivided tetrahedral decomposition from an input point cloud for which a consistent labeling as inside/outside over all tetrahedra is determined by means of a simulated annealing approach. Alliez et al. 
present a variational framework for combined normal direction and orientation estimation as part of their surface reconstruction approach. They first compute a tensor field using a Voronoi diagram of the input point set and derive a best-fitting isosurface by solving a generalized eigenvalue problem. Another variational approach which finds normal directions and orientations simultaneously is presented by Wang et al.. Liu et al.  transfer an input point cloud to a coarse triangulated mesh in order to determine normal orientations on this mesh representing the underlying topology. This information is subsequently used to orient normals on the original point set.
An approach which employs stochastic ray voting is presented by Mullen et al. . An unsigned distance function is first estimated on a 3D Delaunay triangulation of an input point set. Initial estimates for the sign of the distance function are obtained by means of ray shooting and testing for intersections with an -band of the unsigned function which is then smoothed and propagated. Borodin et al.  combine a proximity- and visibility based approach to orient polygons in meshes. A connectivity graph between patches of the model is constructed in which each patch has two visibility coefficients which encode how much of the two sides of the patch is visible from outside. To achieve this, one of the proposed methods is a ray casting approach similar to ours. However, the assumption is that most of the object’s surface is visible from outside the model. Takayama et al.  also employ a ray casting based approach to orient facets in polygon meshes. They cast rays in both directions of facets to determine where outside space is located. For inner facets, they attempt to determine which side of the facet has more free space than the other which is similar to our idea for inner walls. Since this method may fail in cavities, they propose an alternative method based on intersection parity which is prone to modeling errors. In contrast, we employ path tracing with multiple bounces to deal with cavities in the scene. One method that implicitly considers ray paths to propagate inside/outside classifications in triangular meshes is presented by Zhou et al. 
. Based on point samples on triangles, a weighted visibility graph is constructed between the points whose nodes are classified as inside or outside using graph cut.
The input of our approach is a set of points in mainly representing the interior of a building, possibly with some parts of exterior façade and parts of outside area. If (unoriented) normals are not given in the data, they are estimated by means of local principal component analysis (PCA) for each point.
3.1 Plane detection and patch generation
We first detect planes in the point cloud data to obtain a simplified and more structured representation of the scene. Detection of primitive shapes in point clouds is a well studied problem and any reasonable method can be applied. We use the CGAL implementation  of the random sample consensus (RANSAC) method by Schnabel et al.  for its efficiency and quality of the resulting shapes. The rationale behind using planes is that the main structure of buildings can usually be represented well in a piecewise planar manner. Note that other shapes such as spheres or cylinders are also supported by the detection algorithm and could in principle also be used for representing e.g. columns or curved walls.
For each of the detected planes, a relatively coarse 2D occupancy bitmap, i.e. a uniform grid on the surface on which each cell or pixel may have the value or , represents the support of the plane by the points constituting the plane. A pixel of the bitmap has the value if and only if at least one point is located within the pixel. All pixels with value yield the set of patches which will be used in the following steps. Each patch originates from an original surface (i.e. plane) , has a center position and an initial normal with arbitrary but fixed orientation.
3.2 Orientation by path tracing
Our first goal is to estimate an initial normal orientation for each individual patch. Specifically, given a patch , there are two possible orientations for its normal, and . For most points in the datasets we consider, we wish to select the one orientation which points towards the interior of a room. Conversely, the normal should point away from outside area (in case the patch is part of a surface separating room interior and outside area), and away from the interior of wall, floor or ceiling structures (in case the surface separates neighboring rooms).
This classification task is formulated as a voting scheme based on path tracing. Intuitively, for each patch, we trace a number of random paths into both hemispheres for the two possible orientations. We then use the number of ray bounces as well as the path lengths to analyze two aspects. First, we classify whether patches belong to interior or exterior walls, or are located completely outside of the building. Second, we use this classification as well as the path lengths to flip the normal of each patch to the more likely correct orientation. Finally, the reoriented patch normals vote for a normal orientation of whole surfaces.
We now formalize the approach. All ray intersections are tested against the set of patches . Let us first consider a patch with center and one specific orientation . We cast rays , , each with origin and a direction randomly sampled within a cone directed towards . In our experiments, . If a ray with direction intersects with a patch , the ray is reflected into a sampled direction within a cone oriented towards the hemisphere of the incoming ray. The direction of the hemisphere is computed as
Note that the normal of the intersected patch used for this computation is arbitrary but fixed. In particular, the path tracing is invariant under the initial orientation of the patches. We allow up to ray bounces for each initial ray . If a ray does not hit any patch, the respective path is terminated at that point. The result are ray paths, each with up to bounces for the considered patch and orientation .
Let be the length of the th segment along the th path traced for patch and orientation (note that segments after termination of a ray are considered to have zero length). We define the accumulated length as
The rationale for taking the logarithm is to decrease the influence of particularly long segments while still distinguishing between short and medium-length segments. Furthermore, let be the number of bounces of the th path. We consider the average number of ray bounces over all paths
Note that we analogously have and for the opposite direction. We now define a classification function of patch into interior, exterior, or outside as
where is a threshold which was empirically chosen as in our experiments.
A patch with is considered to be clutter outside of the building and subsequently ignored. A patch with is considered to be part of a surface separating room interior from outside area. Its corrected normal orientation is set to point away from the outside area, i.e.
A patch with is considered to be part of a surface between neighboring rooms. In this case, we assume that the orientation with the longer total path length points towards the room interior and we thus set the corrected normal orientation to
The orientation estimation up to this point was performed separately for each patch. Assuming that all points of each of the originally detected planes share a common normal orientation, we can easily vote for an orientation using all patches belonging to a common plane. Let be one of the detected planes with arbitrarily oriented normal and let be the set of patches originating from surface . For voting, we determine the value
where is the standard signum function, and determine the corrected surface normal as
Then all patch normals are flipped to point in the same direction as . For simplicity, we will still call this corrected normal in the following.
3.3 Correction for façade parts
Surfaces belonging to exterior façade are sometimes encountered in interior scans due to scanning through windows. For such patches, the above estimation may erroneously prefer the direction pointing away from the outside area since ray paths towards the outside area are terminated quickly while rays towards the exterior wall of the building generate longer paths. An example for such an erroneous estimation is shown in Figure 3. To correct the orientation in these cases, we perform a second, simpler ray casting pass as follows. For each patch with center and orientation as estimated above, we cast rays , , originating at with directions sampled in a cone oriented towards without allowing ray bounces. Let be the patch which is hit by ray . We then consider the value of
If , the estimated orientation is probably incorrect since it points towards the back side of a surface of a room interior. We thus define the corrected oriented normal as
The patch normals are then again used to vote for a common normal orientation within each surface in the same way as described at the end of Section 3.2.
As a final step, the oriented normals of the patches are used to orient the normals of the original points of the point cloud which lie within the respective patch.
|Dataset 1||Dataset 2||Dataset 3||Dataset 4|
|# points / scans: 5,151,388 / 21||# points / scans: 7,688,111 / 29||# points / scans: 12,409,443 / 13||# points / scans: 34,964,707 / 39|
|points on patches: 73%||points on patches: 71%||points on patches: 73%||points on patches: 83%|
|points non-outside: 68%||points non-outside: 59%||points non-outside: 67%||points non-outside: 80%|
|Plane detection||Plane detection||Plane detection||Plane detection|
|# planes: 228||# planes: 320||# planes: 322||# planes: 556|
|Runtime: 29,320ms||Runtime: 94,468ms||Runtime: 114,929ms||Runtime: 228,903ms|
|Phase 1A: 97.94%||Phase 1A: 96.14%||Phase 1A: 98.08%||Phase 1A: 98.64%|
|Phase 1B: 98.75%||Phase 1B: 97.11%||Phase 1B: 98.05%||Phase 1B: 98.98%|
|Phase 2A: 98.09%||Phase 2A: 98.20%||Phase 2A: 97.53%||Phase 2A: 98.45%|
|Phase 2B: 98.82%||Phase 2B: 98.75%||Phase 2B: 99.42%||Phase 2B: 98.68%|
|Phase 1A: 1885ms||Phase 1A: 3751ms||Phase 1A: 3963ms||Phase 1A: 4142ms|
|Phase 1B: 186ms||Phase 1B: 780ms||Phase 1B: 582ms||Phase 1B: 1462ms|
|Phase 2A: 102ms||Phase 2A: 212ms||Phase 2A: 163ms||Phase 2A: 232ms|
|Phase 2B: 186ms||Phase 2B: 756ms||Phase 2B: 581ms||Phase 2B: 1461ms|
To test the correctness and runtime of our approach, we applied it to multiple real-world datasets with ground truth normal orientations. Specifically, the datasets consist of multiple, registered scans with known scanner positions for each scan. This allows us to flip normals towards the respective scanner positions to obtain the correct orientations. In order to test our approach, we ignore the known orientation and scanner positions, and then compare our estimated orientations with the ground truth. We also measure the runtime of the main processing steps. Table 1 summarizes the results of our experiments which are further discussed below.
4.1 Input data, planes, and patches
The first part of Table 1 shows general statistics about the datasets such as number of points and scans. Note that information about individual scans and scanner positions is only used for generating ground truth normal orientations. It also lists the percentage of the total points which are part of detected planes (and thus belong to patches), and the percentage of the total points which are on patches that are not classified as outside area (i.e. ). Note that it is exactly this set of non-outside points for which our algorithm estimates normal orientations, and that the correctness is measured with respect to this set of points. The number of detected planes and the runtime of plane detection using the RANSAC implementation of the CGAL library  is also listed in the table.
The next part of Table 1 shows the percentage of points on non-outside patches which have been correctly oriented by our algorithm with respect to the ground truth orientations determined using given correspondences of points to known scanner positions. We list the correctness after different phases of our algorithm. Phase 1 is after the initial orientation by path tracing (Section 3.2), before (1A) and after (1B) making normals consistent within surfaces. Phase 2 is after the façade correction step (Section 3.3), again before (2A) and after (2B) ensuring consistency within surfaces.
For each dataset, the images at the bottom of the table show horizontal cross sections of the point clouds for the upper and lower stories. For each story, the upper image shows the unlabeled input point cloud. The middle image shows the classification into interior surfaces (green), exterior (blue), outside (yellow), and points that are not on patches (gray). The lower image shows the final correctness (after phase 2B) of the normal orientation with correctly oriented points (green), incorrectly oriented points (red), outside (yellow), and not on patches (gray).
Figure 1 shows the effect of allowing multiple bounces in our path tracing approach. The images show the orientation correctness after the path tracing phase and before applying surface consistency. Increasing the number of ray bounces helps to correctly identify the orientation of patches which are strongly occluded by surrounding rooms. An overview of the classification of outside area is shown in Figure 2 (a) which shows large areas scanned through windows or from balconies of the building. Outside area is colored yellow. The detail view in Figure 2 (b) shows a cross section of the same building with the more fine grained classification with the same color scheme as in Table 1. An example for façade patches which are initially oriented incorrectly by the path tracing phase is shown in Figure 3 (a). After applying the correction as described in Section 3.3, the normals are oriented correctly (Figure 3 b). A failure case of the façade correction step is shown in Figure 4. The surface highlighted red was incorrectly oriented since the opened, almost parallel door next to it was interpreted as a wall surface. Note that this example is taken from Dataset 4 which explains the decreased final correctness as shown in Table 1.
Below the correctness percentages, Table 1 also lists the runtime of the individual phases of our algorithm as described above. Clearly, the path tracing phase 1A takes more time than the single-bounce façade correction phase 2A. Also, the surface consistency correction 1B and 2B have similar runtimes since they are the same operation performed after phases 1A and 2A, respectively. Even in case of the largest dataset (Dataset 4), the total runtime of the core normal orientation approach takes well below 10 seconds. We are using the NVIDIA OptiX framework  for GPU-accelerated ray tracing against the set of patches which makes the actual ray tracing part a minor part of the total runtime requirements. By far the largest contributor to the overall runtime of our approach is the plane detection for which we currently use a RANSAC implementation in the CGAL library.
5 Conclusion and future work
We have presented a fast and fully automatic approach for orienting normals of the main building structures in multi-room, multi-story indoor point cloud measurements. The input to our algorithm are unstructured point clouds without any additional information such as scanner positions. Using a path tracing approach, we first classify points as interior, exterior, and outside surfaces, and estimate an initial orientation of all non-outside surfaces. In a second phase, we correct the orientation of façade parts which may be incorrectly oriented in the first phase. Additionally, we perform a voting step for consistently orienting normals within surfaces after each phase. We evaluated our approach on multiple, real-world datasets with respect to orientation correctness and runtime. The resulting, automatically estimated orientation information can greatly facilitate or enable tasks such as visualization or reconstruction of building models which rely on correctly oriented surface normals.
While the core of our algorithm provides fast processing of even larger datasets, the overall runtime is strongly dominated by the plane detection. One direction for future work is the evaluation of either different plane detection methods or alternatives for fast patch generation. Also, the façade correction step sometimes incorrectly interprets surfaces as boundaries of rooms and thus performs incorrect flipping of already correct normals. A more sophisticated interpretation of the surfaces may thus be a worthwhile direction for future research.
- Hoppe et al.  Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. Surface reconstruction from unorganized points, volume 26. ACM, 1992.
- König and Gumhold  Sören König and Stefan Gumhold. Consistent propagation of normal orientations in point clouds. In VMV, pages 83–92, 2009.
- Schertler et al.  Nico Schertler, Bogdan Savchynskyy, and Stefan Gumhold. Towards globally optimal normal orientations for large point clouds. In Computer Graphics Forum, volume 36, pages 197–208. Wiley Online Library, 2017.
- Pajarola  Renato Pajarola. Stream-processing points. In Visualization, 2005. VIS 05. IEEE, pages 239–246. IEEE, 2005.
- Murali and Funkhouser  TM Murali and Thomas A Funkhouser. Consistent solid and boundary representations from arbitrary polygonal data. In Proceedings of the 1997 symposium on Interactive 3D graphics, pages 155–ff. ACM, 1997.
- Xie et al.  Hui Xie, Kevin T McDonnell, and Hong Qin. Surface reconstruction of noisy and defective data sets. In Proceedings of the conference on Visualization’04, pages 259–266. IEEE Computer Society, 2004.
- Velho et al.  Luiz Velho, Gabriel Taubin, et al. Estimating the in/out function of a surface represented by points. In Proceedings of the eighth ACM symposium on Solid modeling and applications, pages 108–114. ACM, 2003.
- Alliez et al.  Pierre Alliez, David Cohen-Steiner, Yiying Tong, and Mathieu Desbrun. Voronoi-based variational reconstruction of unoriented point sets. In Symposium on Geometry processing, volume 7, pages 39–48, 2007.
- Wang et al.  Jun Wang, Zhouwang Yang, and Falai Chen. A variational model for normal computation of point clouds. The Visual Computer, 28(2):163–174, 2012.
- Liu and Wang  Shengjun Liu and Charlie CL Wang. Orienting unorganized points for surface reconstruction. Computers & Graphics, 34(3):209–218, 2010.
- Mullen et al.  Patrick Mullen, Fernando De Goes, Mathieu Desbrun, David Cohen-Steiner, and Pierre Alliez. Signing the unsigned: Robust surface reconstruction from raw pointsets. In Computer Graphics Forum, volume 29, pages 1733–1741. Wiley Online Library, 2010.
- Borodin et al.  Pavel Borodin, Gabriel Zachmann, and Reinhard Klein. Consistent normal orientation for polygonal meshes. In Computer Graphics International, 2004. Proceedings, pages 18–25. IEEE, 2004.
- Takayama et al.  Kenshi Takayama, Alec Jacobson, Ladislav Kavan, and Olga Sorkine-Hornung. A simple method for correcting facet orientations in polygon meshes based on ray casting. Journal of Computer Graphics Techniques, 3(4):53, 2014.
- Zhou et al.  Kaichi Zhou, Eugene Zhang, Peter Wonka, et al. Visibility-driven mesh analysis and visualization through graph cuts. IEEE Transactions on Visualization and Computer Graphics, 14(6):1667–1674, 2008.
- Oesau et al.  Sven Oesau, Yannick Verdie, Clément Jamin, Pierre Alliez, Florent Lafarge, and Simon Giraudot. Point set shape detection. In CGAL User and Reference Manual. CGAL Editorial Board, 4.12 edition, 2018. URL https://doc.cgal.org/4.12/Manual/packages.html#PkgPointSetShapeDetection3Summary.
- Schnabel et al.  Ruwen Schnabel, Roland Wahl, and Reinhard Klein. Efficient ransac for point-cloud shape detection. Computer Graphics Forum, 26(2):214–226, June 2007.
- Parker et al.  Steven G Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, et al. OptiX: a general purpose ray tracing engine. ACM Transactions on Graphics (TOG), 29(4):66, 2010.