Consider two cameras and observing the same 3D point . If the internal calibration and the relative pose of the two cameras are known, we can backproject the measured point in each image and obtain the rays from each camera, pointing to . Now, we define the normalized epipolar error as follows:
where and are the backprojected unit rays from and , respectively, is the rotation matrix and
is the translation vector that together transform a point from the reference frameto , i.e., and . The second equality in (1) follows from the fact that the scalar triple product is invariant to a circular shift. In the literature, the error is often expressed as follows:
where is the essential matrix and
is the skew-symmetric operator.
If the image measurements, calibration and pose data are all perfectly accurate, this error would be zero because , and would be coplanar (see Fig. 1). This is called the epipolar constraint . In practice, the raw data contain inaccuracies, so they do not satisfy this constraint most of the time. For this reason, many existing works in 3D vision try to solve geometric reconstruction problems by minimizing the cost based on this error [1, 2, 4, 5, 10, 12, 16]
and/or using it to identify outliers[11, 16].
In the literature, the normalized epipolar error has mostly been treated as an algebraic quantity that has no geometric meaning[1, 2, 4, 5, 12, 14, 16]. We believe that this misconception stems from the fact that the “standard” epipolar error is an algebraic quantity [3, 9, 13, 15]:
where and are the normalized image coordinates of the point in and , respectively. Notice that the only difference between (1) and (3) is the way the rays are normalized: In (1), they are normalized by their lengths, whereas in (3), they are normalized by the last element in the vector.
In , a geometric interpretation was given for the following error:
which corresponds to the cosine of the angle between and . This is equal to the perpendicular distance between the point at and the plane containing and .
In this work, we provide geometrically intuitive interpretations of (1) by relating it to the following quantities:
The volume of the tetrahedron where , and form the three edges meeting at one vertex.
The shortest distance between the two backprojected rays and .
The dihedral angle between the two bounding epipolar planes, , one plane containing and and the other containing and .
The -optimal angular reprojection error.
2 Geometric Interpretations of (1)
1. Relation to the volume of a tetrahedron:
Consider the tetrahedron shown in Fig. 2reda. One of its vertices is placed at (, the position of camera , which is the origin in the reference frame of ), and the other three at , and . Then, using the well-known formula for the volume of a tetrahedron, its volume is obtained by
The nice thing about this interpretation is that it allows for a simple visualization of the error, as shown in Fig. 2reda. As the degree of coplanarity increases among the three edges (, and ), the common vertex will be “pulled” towards the opposite side, flattening the tetrahedron. When the three edges are coplanar, the tetrahedron becomes completely flat, , , and thus .
2. Relation to the distance between the two rays:
We can also relate the normalized epipolar error (1) to the shortest distance between the two backprojected rays, , in Fig. 2redb. To show this, we will first derive the formula for the shortest distance between two skew lines. Consider two skew lines and . The distance between them is given by the distance between the closest pair of points on each line ( and ), and they lie on the common perpendicular to both lines111We can easily prove this by contradiction. We omit the proof.. Now, consider two parallel planes with the normal : plane containing and plane containing , as illustrated in Fig. 3reda. Notice that is the same as the distance between the planes, which is the same as where is the projected position of in . Since , we get
This means that in Fig. 2redb is given by
Let be the angle between and (also known as the raw parallax angle ), ,
Then, (8) can be written as
Therefore, we can interpret as the distance between the two backprojected rays, weighted by
. For relative pose estimation between two views, we can assumewithout loss of generality, so minimizing the cost based on (10) is equivalent to minimizing the cost based on . We can interpret as the factor that downweights the residual when the parallax angle is small. Note that and the equality holds if and only if and are both perpendicular to . If the two rays intersect (at infinity), then (or ), and thus .
As a side note, it should be mentioned that given by (8) is the distance between the lines rather than the rays. Technically speaking, it is the shortest distance between line for and line for .
3. Relation to the angle between the two planes:
In Fig. 2redb, consider the following planes: one plane containing and , and another containing and . Let and be their normal vectors. These two planes are drawn in Fig. 3redb. We can think of them as the two bounding planes between which the epipolar plane is usually found. This is the case for most two-view triangulation methods (, midpoint methods  and optimal methods ). The dihedral angle between these two bounding planes is given by
This can be rearranged to
For any 3D vector , , , , the vector quadruple product is equal to . Therefore, the right-hand side of (12) can be written as
These two angles are shown in Fig. 2redb. From (16), we can interpret as the sine of the dihedral angle between the two bounding epipolar planes, weighted by . Therefore, would be small if either of , or is very small. This makes sense because the epipolar geometry degenerates as or approaches zero. Also, when is small, the two bounding epipolar planes are close to coplanarity, and so do the vector , and .
4. Relation to the optimal angular reprojection error:
The -optimal angular reprojection error  is defined as follows:
Rearranging this, we get
Therefore, we can interpret as the sine of the -optimal angular reprojection error, weighted by . It follows that would be small if either of or is very small. This makes sense because small means that only a little correction is needed for the two backprojected rays to intersect. Also, small means that the vector , and are all close to parallelism, which brings the epipolar geometry close to degeneracy. What may seem peculiar in (22) is the fact that does not reflect the degeneracy when either or is zero. However, this is not an issue, because the term is necessarily zero whenever degeneracy occurs. In the Appendix, we verify (22) using simulation.
In this work, we presented several geometric interpretations of the normalized epipolar error defined in (1). Specifically, we revealed the direct relations between this error and the following quantities:
This work was partially supported by the Spanish government (project PGC2018- 096367-B-I00) and the Aragón regional government (Grupo DGA-T45_17R/FSE).
The contributions of this work are the derivations of (6), (10), (16) and (22). No approximation is made in the derivations, so strictly speaking, experiments are redundant as long as the mathematics are correct. Having said that, we understand that some readers may have doubts about the derivations, and also, it is essential to verify the theoretical results whenever possible (as a sanity check). In the case of (6), (10) and (16), however, performing experiments is pointless because the only sensible method to compute the volume , the distance and the angle is to use the very same formulas used in the derivations. For this reason, we only focus on the verification of (22) in this section.
In simulation, we create two cameras and one point. The two cameras are placed at position and where is a random 3D vector of length 0.5 unit and . This ensures that unit. The image size is set to pixel and the focal length to pixel. We place the point at where
follows the uniform distribution. Then, we orient the cameras randomly until the point is visible in both views. The image coordinates of the projected point are perturbed by Gaussian noise with pix.
We correct the backprojected rays using the -optimal triangulation method described in  and obtain the angular error using (19). To check if this is locally optimal, we perturb the resulting 3D point by small random noise and see if we achieve smaller error. We set the noise magnitude to unit with , and for each magnitude, we perturb the point one hundred times independently.
We repeat this procedure times and aggregate the results. All computations are done in Matlab. Fig. 5 shows the histograms of the normalized epipolar error computed using (1) before and after the -optimal ray correction . Comparing the two histograms, we see that the corrected rays do intersect. Fig. 6 presents the result of the perturbation test. It shows that the angular error of the corrected rays is (locally) minimum within the numerical accuracy. Plugging into (22), we obtain an estimate of , , . In Fig. 7, we plot the histogram of the absolute difference . Notice that it is as small as the normalized epipolar error of intersecting rays (see Fig. 5). Therefore, we can safely conclude that within the numerical accuracy. In Fig. 8, we provide, for completeness, the histogram of the relative difference .
J. Briales, L. Kneip, and J. Gonzalez-Jimenez.
A certifiably globally optimal solution to the non-minimal relative
IEEE Conf. Comput. Vis. Pattern Recognit., pages 145–154, 2018.
-  M. Garcia-Salguero, J. Briales, and J. Gonzalez-Jimenez. Certifiable relative pose estimation. CoRR, abs/2003.13732, 2020.
R. Hartley and A. Zisserman.
Multiple View Geometry in Computer Vision. Cambridge University Press, New York, NY, USA, 2 edition, 2003.
-  U. Helmke, K. Hüper, P. Y. Lee, and J. Moore. Essential matrix estimation using gauss-newton iterations on a manifold. Int. J. Comput. Vis., 74(2):117–136, 2007.
-  L. Kneip and S. Lynen. Direct optimization of frame-to-frame rotation. In IEEE Int. Conf. on Comput. Vis., pages 2352–2359, 2013.
-  S. H. Lee and J. Civera. Closed-form optimal two-view triangulation based on angular errors. In IEEE Int. Conf. Comput. Vis., pages 2681–2689, 2019.
-  S. H. Lee and J. Civera. Triangulation: Why optimize? In Brit. Mach. Vis. Conf., 2019.
-  H. C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293(5828):133–135, 1981.
-  Q. Luong and O. D. Faugeras. The fundamental matrix: Theory, algorithms, and stability analysis. Int. J. Comput. Vis., 17(1):43–75, 1996.
-  A. Pagani and D. Stricker. Structure from motion using full spherical panoramic cameras. In IEEE Int. Conf. on Comput. Vis. Workshops, pages 375–382, 2011.
-  S. H. Lee and J. Civera. Robust uncertainty-aware multiview triangulation. CoRR, abs/2008.01258, 2020.
-  A. L. Rodríguez, P. E. López-de-Teruel, and A. Ruiz. Reduced epipolar cost for accelerated incremental SfM. In IEEE Conf. Comput. Vis. Pattern Recognit., pages 3097–3104, 2011.
-  P. Torr and D. Murray. The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Comput. Vis., 24(3):271–300, 1997.
-  J. Yang, H. Li, and Y. Jia. Optimal essential matrix estimation via inlier-set maximization. In Eur. Conf. Comput. Vis., pages 111–126, 2014.
-  Z. Zhang. Determining the epipolar geometry and its uncertainty: A review. Int. J. Comput. Vis., 27(2):161–195, 1998.
-  J. Zhao. An efficient solution to non-minimal case essential matrix estimation. CoRR, abs/1903.09067, 2019.