Log In Sign Up

Model Quality Aware RANSAC: A Robust Camera Motion Estimator

Robust estimation of camera motion under the presence of outlier noise is a fundamental problem in robotics and computer vision. Despite existing efforts that focus on detecting motion and scene degeneracies, the best existing approach that builds on Random Consensus Sampling (RANSAC) still has non-negligible failure rate. Since a single failure can lead to the failure of the entire visual simultaneous localization and mapping, it is important to further improve robust estimation algorithm. We propose a new robust camera motion estimator (RCME) by incorporating two main changes: model-sample consistence test at model instantiation step and inlier set quality test that verifies model-inlier consistence using differential entropy. We have implemented our RCME algorithm and tested it under many public datasets. The results have shown consistent reduction in failure rate when comparing to RANSAC-based Gold Standard approach. More specifically, the overall failure rate for indoor environments has reduced from 1.41


page 1

page 2

page 3

page 4


Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes

Long-term camera re-localization is an important task with numerous comp...

A Survey of Structure from Motion

The structure from motion (SfM) problem in computer vision is the proble...

Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments

Localizing the camera in a known indoor environment is a key building bl...

Beyond Camera Motion Removing: How to Handle Outliers in Deblurring

Performing camera motion deblurring is an important low-level vision tas...

Camera motion estimation through planar deformation determination

In this paper, we propose a global method for estimating the motion of a...

Exploiting Points and Lines in Regression Forests for RGB-D Camera Relocalization

Camera relocalization plays a vital role in many robotics and computer v...

1 Introduction

Robust estimation of geometric relationship between two camera views is a fundamental problem in computer vision and robotics. It simultaneously identifies corresponding inlier features from outlier noises. When applying to the problem of visual odometry (VO) or visual simultaneous localization and mapping (vSLAM) in robotics, the geometric relationship is often the fundamental matrix, or essential matrix when camera intrinsic parameters are known. Since camera motion can be inferred from essential matrix, this is also known as camera motion estimation.

Figure 1: The system diagrams of our RCME framework in (b) which is an improvement of RANSAC in (a), the Gold standard algorithm for estimating camera motion.

The classical robust estimation method that can filter out outliers is random consensus sampling (RANSAC) [9] (Fig. 1). Simply applying RANSAC to estimate camera motion is not wise since there are well-known motion degeneracy and scene degeneracy issues, also known as Type A and Type B degeneracies according to [30], respectively. To eliminate both degeneracies, existing state-of-the-art approaches apply key frame selection and simultaneously estimate homography and fundamental matrices [19]. However, these remedies do not solve all the problems. After testing 14816 images from 23 different datasets, our results show that the RANSAC-based fundamental matrix estimation algorithm still has a failure rate of in indoor environments, despite that these test data have already both degeneracies removed. Since one single failure can lead to an entire continuous monocular vSLAM failure, this leads to the mean distance between failure is mere 35 meters if the average distance between adjacent key frames is 0.5 meters. This means that current robust estimation of camera motion is not satisfying.

A deep look into the failed cases reveals that due to the fact that the epipolar geometric relationship characterized by a fundamental matrix is a weak point-to-line distance measure and scene feature distribution may not be uniform, it is possible that the initial sampling in RANSAC may establish a wrong model and still can find a potential large number of inliers. As a result, RANSAC may output wrong camera motion. To address these issues, we propose a new robust camera motion estimator (RCME) (see Fig. 1) by incorporating two main changes: model-sample consistence test to verify the quality of the model built by over-parameterized sampling, and inlier set quality test that first determines inliers according to model uncertainty and then verifies if the inlier set agrees with the model through a test on differential entropy. The RCME can also detect and report a failed estimation when input data quality is too low.

We have implemented the RCME algorithm in C++ and tested with a variety of public datasets. The results have shown consistent reduction of failure rate across almost all test data. More specifically, the overall failure rate for indoor environments has reduced from to .

2 Related Work

Robust camera motion estimation mainly relates to two research fields: robust estimation in computer vision and the front-end algorithm of vSLAM/VO in robotics.

Robust estimation is a fundamental problem in computer vision and robotics. A robust estimator’s task is to estimate parameters and find the inliers from noisy data correspondences according to a predefined type of geometric relationship. The outliers from wrong correspondences often introduce significant errors when estimating the geometric relationship. M-estimator, L-estimator and R-estimator [14] formulate the estimation problem as a reprojection error minimization problem and solve it by using a nonlinear cost function. Leat Median of Squares (LMeds) by Rousseeuw [25] minimizes the median of error instead. These methods are not stable when over half of data are outliers. RANSAC by Fischler et al. [9] is the most widely used robust estimator since it is capable of handling a high ratio of outliers.

The vSLAM and VO works can be classified as the feature-based approaches 

[5, 16, 19], the semi-dense approaches [8, 10], and the direct approaches [7, 6]. Among these methods, the widely adopted feature-based approaches explicitly and repeatedly utilize robust estimation of camera motion (e.g., MonoSLAM [5] and PTAM [16]). RANSAC has been employed in combination with different fundamental matrix estimation models such as the one-point method in [27], the five-point algorithm [28] in PTAM or the eight-point method in ORB-SLAM2  [19]. These works repeatedly obtain initial solutions for camera poses (key) frame by (key) frame before bundle adjustment. Hence the robustness of the robust estimation method is critical here because a single failure can collapse the entire mapping process which many involve thousands of RANSAC-based camera motion estimation instances.

Our work is an improvement over the existing RANSAC framework for camera motion estimation. In fact, most existing works in this area mainly focus on detecting the two type of degeneracy issues: Type A and Type B [30]. When handling degeneracy cases, existing approaches often treat the problem as a model selection between the homography relationship and the fundamental matrix relationship. Popular approaches include AIC [1], PLUNDER-DL score [32], GIC [15], and GRIC [29]. Pollefeys et al. [22] uses GRIC in recovering the structure and motion. ORB-SLAM2 [19]

initializes the trajectory by finding the better geometry model between the fundamental matrix or the homography matrix. However, even when Type A and Type B degeneracies have been ruled out, our tests have shown that RANSAC-based camera motion estimation may still fail with non-negligible probability. It is due to the fact that the fundamental matrix only represents a weak geometric point-line distance geometric relationship which is not very selective.

Many attempts have been tried to improve RANSAC in general. The loss function function 

[31] is introduced in model selection. Choi et al. [4] also provides a survey to RANSAC-based methods and categorizes them into three groups: being fast, being accurate, and being robust. Our work emphasizes being robust. Inspired by the unified framework named USAC [23], our work employs the information entropy for the hypothesis evaluation to improve model quality and hence increase robustness. The threshold used in determined inliers is based on the model uncertainty. The later step is inspired by Cov-RANSAC [24] which also incorporates the model uncertainty to identify inliers, but they do not rank the model by using the model-inlier consistence.

3 Problem Definition

We begin with the following assumptions:

  • The camera is pre-calibrated and its lens distortion is removed from images.

  • Position noises of the points follow zero-mean Gaussian distribution with known variance

    in each dimension and the noise in each dimension is independent.

Let us define the common notations in this paper.

the intrinsic matrix of the camera.

is a

-vector and concatenates point correspondences from two views. The

-th point is denoted as , where is the -th corresponding points from the first and the second views, respectively. Symbol indicates the inhomogeneous coordinate and . The corresponding 3-vector homogeneous representation is .

is a -vector with entries from fundamental matrix , , where denotes the -th row of . The -th model vector denotes as .

is a -vector in defining the camera motion and consists of a unit quaternion vector and a translation vector . The -th camera motion denotes as .

In this paper, denotes the identity matrix.

3.1 Inputs

We pre-process both views to obtain inputs. Feature detection and feature matching have been applied to obtain putative point correspondences , where is the number of point correspondences. We concatenate every inhomogeneous point correspondence in to a -vector . The inputs may be obtained from a variety of feature detectors such as scale-invariant feature transform (SIFT) [17], speeded up robust feature (SURF) [2], or ORB [26]. It is worth noting that the point correspondence set is the result of putative matching of feature descriptors and often contains many outliers.

3.2 Problem Definition

With notations defined and inputs introduced, our problem is defined as,

Definition 1

Given point correspondences (or points for brevity) , determine if the solution of the camera motion exists. If the solution exists, estimate the camera motion .

As shown in the problem definition, one immediate difference between our method and RANSAC is that our method can detect failure cases instead of output an unreliable solution. This is actually very useful for vSLAM because it can be used as a signal to re-adjust key frame selection.

4 RCME Aglorithm

4.1 RANSAC Review and RCME Overview

Since our RCME is an extension of RANSAC, let us begin with a brief review of RANSAC as shown in Fig. 1. RANSAC establishes hypothesis models by randomly sampling a minimal correspondence set (model instantiation) and then examines each set by comparing how many other features agree with the hypothesis model (model verification). RANSAC is an iterative method which terminates when reaching a maximum iteration number or a high quality inlier set with its size agreeable to the estimated inlier ratio is found. The model with the largest inlier set is selected as the output.

Building on basic RANSAC framework, Fig. 1 illustrates our system diagram. It contains four main blocks in gray. The first three blocks are different from the counterpart. After instantiating a hypothesis model through sampling, we test the consistence between the samples and the model. If no consistence exists between the samples and the model, we discard the model and repeat the model instantiation step. Otherwise, we begin model verification, where we perform an inlier quality test before we select the model according to entropy. If no model can be selected, we consider the camera motion estimation as failure in termination step (See from Box 3(b) in Fig. 1). The selected model is our initial solution for the model refinement.

Without loss of generality, let us assume we are at the -th iteration to begin the explanation.

4.2 Model Instantiation

Model instantiation is the first step of every iteration. We want to check if a model building on the randomly sampled point correspondences actually agrees with the samples. This is crucial for overpameterized models.

Same as the traditional RANSAC, we randomly sample point-correspondences from where is the minimal number of samples to instantiate the model and can be different according to different parameterizations used in modeling. Define the sample set for the -th iteration as


where is the point index and denotes the -th sample.

4.2.1 model instantiation and uncertainty analysis

Given the sample set , we instantiate the model and recover the camera motion from . Consider a mapping function which maps the model to the concatenated samples , . The concrete representation of depends on parameterization of fundamental matrix which includes point [27], point [20], or point methods [13]. Since our framework is not limited by a particular parameterization, we represent it as a generic mapping.

The model instantiation of fundamental matrix can be represented by


The camera motion is recovered from the fundamental matrix decomposition [12]. The camera motion and the fundamental matrix model always satisfy


where is a scalar,

is the skew-symmetric matrix representation of

, and is the rotation matrix of the unit quaternion vector . Let be the implicit function between and ,


Ultimately, the uncertainty of depends on the error distribution of model instantiation samples and is propagated through the mapping function and . The noise distribution of is modeled as a zero-mean Gaussian with the covariance matrix according to the assumption a.2.

Before estimating the covariance matrix of to characterize its uncertainty, it is necessary to know how fundamental matrix estimation is parameterized:

  • Exact case

    : The fundamental matrix is parameterized by the same amount of parameters as the degrees of freedom (DoFs) of fundamental matrix. For example, the 1-point algorithm is an exact case since the fundamental matrix is parameterized by one yaw angle and the DoF of fundamental matrix is

    when the robot motion is assumed to follow the Ackermann steering model on a planar surface.

  • Overdetermined case: The amount of parameters used to parameterize the fundamental matrix is larger than the DoFs of fundamental matrix. For example, the normalized 8-point algorithm is a common overdetermined case since it employs parameters but the DoFs of a general fundamental matrix is .

The choice of parameterization method affects how we estimate the uncertainty.

We utilize the first order approximation [12] of the covariance matrix to estimate the uncertainty. Denote the covariance matrix of as . When the fundamental matrix estimation is the exact case, the first order approximation of is


where Jacobian matrix . For the overdetermined case, let us use the point method fundamental matrix estimation as an example. We have


where is a matrix and its column vectors are perpendicular to . We obtain by using the first columns of the Householder matrix of .

The uncertainty of depends on the uncertainty of and . Denote the covariance matrix of as . Under the Gaussian noise assumption, the first-order approximation of is


where Jacobian matrix can be obtained either by explicitly where symbol is matrix pseudo inverse, or calculating from the fundamental matrix decomposition [21].

4.2.2 Samples and consistence test

For the over-parameterized case, a good model must be consistent with the samples that instantiate it. Therefore, we verify if it is the case. Only the model which passes this consistence test can advance to the next step. Otherwise, we discard the model and the -th iteration ends. Of course, this does not apply to the exact case where its model always perfectly fits the samples.

The consistence between the samples and the model is measured by the error distance. We use the Sampson error vector [12] to form the error measurement. Denote the Sampson error vector of as . Let as the Sampson error vector function which utilizes to calculate the Sampson correction of


We model as the zero-mean Gaussian distribution with the covariance . Under the Gaussian noise assumption, the first-order approximation of is


where Jacobian matrices and .

For each sample , we design the following hypothesis testing:


Given the Sampson error vector (8) and the covariance matrix (9), the error distance is re-written as


Since we approximate

as the normal distribution with zero mean vector and the covariance

, (11) is a distribution. Besides, is defined on the variety of , which reduces DoF. Therefore, (11) follows a distribution with DoFs. Define as the cumulative distribution under DoFs and we can set the distance threshold by setting the significance level , where is the inverse function of . Thus, we consider agrees with by rejecting when


4.3 Model Verification

For the model which passes the aforementioned consistence tests, we find inliers from the rest of the inputs and verify the model by checking the quality of its inliers (See Boxes 2(a) and 2(b) in Fig. 1).

4.3.1 Find inliers

An inlier is defined to be a point consistent with the model. We employ consistence test to find the inliers.

For the point , the Sampson error vector and corresponding covariance are denoted as and , respectively, and can be obtained from (8) and (9). Let as an inlier indicator function of


The inlier set of is

with its size defined as .

4.3.2 Score inliers

It measures the quality of the consistence between model and its inlier set. Instead of using , the number of inliers or the loss function [31] to score the inliers, we want to score inliers by using the differential entropy on the covariance matrices

. The intuition is that the joint distribution of distances

for the inliers should have a small entropy for a high quality inlier set. For inlier set , we define a score vector as,


where each entry is a differential entropy for each inlier,


where denotes the determinant of .

4.3.3 Inlier set quality test

Now, we evaluate the inlier’s quality to determine if the model can enter candidate solution set by checking entropy values. A good model must contain the inliers with high quality of consistence and leads to small entropy values. Given the score vector

, the average entropy and the standard deviation are defined as follows:


where is L norm, is L norm, and is a n-vector of ones.

To evaluate the quality of the model and its inlier set, we design the following hypothesis testing based on the test,



is an differential entropy threshold determined by the experiments. The test statistic can be calculated



as the cumulative distribution function of the standard normal distribution at value

. By setting the significance level , the p-value is obtained We consider that the model is highly consistent with its inlier set by rejecting when


For models that passed the hypothesis testing, we proceed to next step.

A good model must contain both sufficiently large amount of and high quality inliers. We use two ratio thresholds and to determine if the -th model satisfies the requirement. is a priorly-known or estimated inlier ratio. A model is considered to contain a sufficiently large amount of inliers when


where conservative coefficient determine how close to ideal size we want the inlier set to be. Same as the RANSAC, inlier ratio can be inferred in the process. All models that survive the hypothesis testing in (LABEL:eq:inlier-test) and size condition in (20) are added to the candidate solution set,


After the maximum number of trials , we consider the camera motion recovery fail if . This leads to the failed algorithm output (i.e. ‘’ in Fig. 1). The failure reason can be either that the iteration number is not big enough or that the input is from poor quality key frames.

4.4 Model Selection and Early Termination

4.4.1 Model selection

For non empty , we select the one with the minimum average entropy to be the output.


This output serves as an initial solution for the following model refinement by applying Maximum Likelihood Estimation (MLE) to minimize reprojection error. Since this is the same as the traditional approach (i.e. Box 4s in both Fig. 1 and Fig. 1, we skip it here.

4.4.2 Early termination condition

So far, our RCME algorithm runs for the entire maximum iterations. It is possible to design an early termination threshold to speed up the algorithm. Note that entries in grow after each successful iteration. For each new entry, we can test its average entropy by comparing to a preset threshold. If it is small enough, which means it is a satisfying solution, we can terminate RCME early to perform the MLE-based model refinement.

5 Algorithm and Complexity Analysis

Input : 
Output : Fail or {, }
Initialize maximum iteration number ;
Initialize the model ranking ;
for  to  do
       Select minimal sample set ;
       Estimate and using (2) and (4);
       if  is overdetermined then
             Perform and consistence test (10);
             if  fails in consistence tests then
      Find and score inliers using (13) and (14);
       Perform inlier quality tests (17);
       if  fails in inlier quality test then
      Update the candidate set by (21);
       if  satisfies early termination (Sec. 4.4.2) then
if  then
       return Fail
Select from using (22);
Perform MLE on ;
return ,
Algorithm 1 RCME algorithm

Our RCME algorithm is summarized in Algorithm 1 to facilitate our complexity analysis. Each iteration begins with the model instantiation and it takes to select samples from points by using pseudo-random number generator algorithm [33]. Model estimation takes time. Performing the model consistence test on every sample takes since is a small and fixed constant. It takes time to determine whether each point is an inlier or not. Updating the model candidate takes constant time to check if the model passes the inlier quality test and satisfies the size condition. The overall computational complexity for each iteration is . The total iteration performed in RCME is bounded below . Thus, the overall iteration process takes .

Suppose on average models enter the final candidate set, where . Therefore, model selection takes . MLE is solved by applying the Levenberg-Marquardt (LM) algorithm [12]. In each iteration of LM, the dense matrix solver takes . The total iteration needed by LM is bounded below with the stopping threshold . The overall computational complexity for re-estimation is . To summarize the analysis, we have the following,

Theorem 1

The computational complexity of our RCME algorithm is .

6 Experiments

6.1 Algorithms in Comparison and Settings

We have implemented our system in C++. We evaluate algorithm robustness using frequently-used public data sets. We compare RCME with the Gold standard approach with RANSAC for computing the fundamental matrix [12] which is the most widely-used fundamental matrix estimation method. We abbreviate the the Gold standard approach as “Standard” in the comparison. We also include “pRCME” which is RCME with model consistence test (10) in Sec. 4.2.2 turned off. The purpose is to show the individual effectiveness of the consistence test and the following inlier quality test.

All algorithms employ the normalized -point algorithm [12] for the fundamental matrix estimation since it is fast and works for general camera motion. The maximum iteration number is set to . We perform a complete iterations instead of estimating during the iteration process. This follows the setup in ORB-SLAM2 [19]. The noise is set to .

6.2 Testing Datasets

We test our method with a wide range of public datasets: KITTI odometry dataset [11], EuRoC MAV dataset [3], and HRBB 4th floor dataset in Texas AM University (HRBB4) [18]. We divide test datasets into two groups: indoor datasets and outdoor datasets because their feature distributions are quite different.

Indoor datasets include HRBB4 and EuRoC MAV. The datasets cover different indoor environments. The HRBB4 dataset is recorded in the office corridor environment. The EuRoC MAV dataset is collected by a synchronized stereo camera. Since this paper only concerns monocular vision, we only utilize the images from the left camera. The EuRoC MAV dataset contains two types of indoor environments, one is recorded in a machine hall and the other is recorded in a vicon room. The EuRoC MAV dataset has sequences with different types of camera motion and lighting conditions. It covers from office rooms to industrial halls with different objects and camera motions.

The outdoor dataset includes KITTI odometry dataset. The KITTI odometry dataset is recorded by a moving vehicle while driving in Karlsruhe, Germany. We perform tests on image sequences of the KITTI odometry dataset.

In all cases, the sequence of two-view image pairs of each dataset are initially selected by applying the initialization module (i.e. key frame selection) of ORB-SLAM2 [19] and we only keep the non-degenerated two-view image pairs. For each two-view image pair, we use SIFT [17] to obtain the point correspondences as our inputs.

6.3 Evaluation Metric for Robustness

Robustness is measured by failure rate of each algorithm. It is important to recognize a failed case.

For every two-view pair, we use the ratio of the consistent inlier amount before and after performing MLE in model refinement step (Box 4s in Fig. 1 and Fig. 1) to identify if the method fails to find a correct solution. Let and be the consistent inlier amount before and after MLE, respectively. A quality solution should cause the number of inliers to increase or at least maintain its inlier set size instead of decreasing inlier set size drastically. The underlying rationale is that an incorrect solution usually falls into local minima instead of the solution close to the global minima in the optimization process. The local minima causes the solution loses the consistence from its inlier data.

We employ the Huber robust function on top of reprojection error (page 617 in [12] ) to re-evaluate the consistence of inliers before and after MLE to get and , respectively. An inlier remains consistent with its model when the value of the Huber robust function is less than the threshold , where denotes the cumulative distribution under DoF and is the significance level.

We set a ratio threshold to determine that the solution is considered as a failed camera motion recovery when


We collect the overall failure rate for each dataset as the measure for robustness for the dataset.

Note that precision that is represented by residual error in cost function is not the concern here because both standard RANSAC and RCME can output high quality solutions in precision if they do NOT fail. In other words, precision is useless if they fail to find the correct solution.

It is worth noting that we do not use the provided “ground truth” from some datasets because those “ground truth” were mostly computed from the Gold Standard approach and its correctness is not well verified. Only a small ratio of wrong ground truth would cause comparison issues.

6.4 Experimental results

The experimental results are shown in Tab. 1. The upper half of the table are results from outdoor datasets and the lower half of table are results from indoor datasets.

For both RCME and pRCME, they have the ability to detect poor quality inputs. In those case, the algorithms do not proceed and directly output failure which are represented as “Detect-fail” columns. The “Failure” columns represent the actual failure computed using the metric in Sec. 6.3. The “Failure” columns do not include cases in “Detect-fail” columns. In general, we want the failure rate to be as close to zero as possible. It is clear that both RCME and pRCME outperform the “Standard” approach, even you may add “Detect-fail” column with “Failure” column together and compare the values to those of the “Standard” approach. Our algorithms have improved robustness of RANSAC for camera motion estimation.

It is also clear that RCME is better than pRCME, which indicates that our model consistence test with its samples works as expected. Also, the fact that pRCME is better than the “Standard” approach means that our inlier-quality test works as expected. It is worth noting that the performance improvement is more significant for indoor cases. This is true because indoor datasets often suffer more from uneven feature distribution and poor light conditions. This means RCME handles those challenging conditions better. More specifically, the average failure rate for indoor datasets is reduced from to .

RCME pRCME Standard
Dataset Detect-fail % Failure % Detect-fail % Failure % Failure %
KITTI/00(#2412) 0.00 0.00 0.00 0.00 0.00
KITTI/01(#56) 0.00 0.00 0.00 0.00 1.79
KITTI/02(#2031) 0.00 0.00 0.00 0.05 0.05
KITTI/03(#358) 0.00 0.00 0.00 0.00 0.00
KITTI/04(#86) 0.00 0.00 0.00 0.00 0.00
KITTI/05(#1489) 0.00 0.00 0.00 0.00 0.27
KITTI/06(#295) 0.00 0.00 0.00 0.00 0.00
KITTI/07(#564) 0.00 0.00 0.00 0.00 0.00
KITTI/08(#1690) 0.00 0.00 0.00 0.00 0.18
KITTI/09(#678) 0.00 0.00 0.00 0.15 0.00
KITTI/10(#419) 0.00 0.00 0.00 0.00 0.24
Avg. 0.00 0.00 0.00 0.02 0.10
EuRoC/MH_01(#515) 0.19 0.00 0.00 0.19 0.19
EuRoC/MH_02(#450) 0.00 0.00 0.00 0.00 0.89
EuRoC/MH_03(#587) 0.00 0.00 0.00 0.00 0.00
EuRoC/MH_04(#357) 0.28 0.00 0.28 0.00 0.84
EuRoC/MH_05(#393) 0.25 0.00 0.00 0.00 0.25
EuRoC/V1_01(#566) 0.00 0.00 0.00 0.00 1.77
EuRoC/V1_02(#383) 0.26 0.00 0.26 0.00 1.83
EuRoC/V1_03(#108) 0.93 0.00 0.93 0.00 8.33
EuRoC/V2_01(#391) 0.26 0.00 0.26 0.26 2.81
EuRoC/V2_02(#590) 0.00 0.17 0.00 0.17 3.05
EuRoC/V2_03(#92) 0.00 0.00 0.00 0.00 3.26
HRBB4th(#306) 0.00 0.00 0.00 0.00 0.33
Avg. 0.13 0.02 0.08 0.06 1.41
Table 1: Experimental Results. Best results are highlighted in boldface.

7 Conclusions and Future Work

We reported our new robust camera motion estimation algorithm targeted at improving robustness in traditional RANSAC-based approaches. Combining two new developments: model quality test with its samples, and inlier quality test with its model, we are able to consistently reduce failure rate of the existing algorithm, as shown in the experimental results from testing a wide range of indoor and outdoor datasets.

In the future, we will look deep into those failed cases that have not been detected by our algorithm and try to design new tests to further improve the robustness of the entire algorithm. We are in the processing of embedding our estimator into existing open source implementation such as ORB-SLAM2 to share the new developments with others.


We thank Y. Xu for his insightful discussion. We are also grateful to H. Cheng, B. Li, A. Kingery, A. Angert, D. Wang, and Y. Ou for their inputs and contributions to the Networked Robots Laboratory at Texas A&M University.


  • [1] Hirotugu Akaike. A new look at the statistical model identification. IEEE transactions on automatic control, 19(6):716–723, 1974.
  • [2] H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded up robust features. In European Conference on Computer Vision, pages 404–417. Springer, 2006.
  • [3] Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart. The euroc micro aerial vehicle datasets. The International Journal of Robotics Research, 35(10):1157–1163, 2016.
  • [4] Sunglok Choi, Taemin Kim, and Wonpil Yu. Performance evaluation of ransac family. Journal of Computer Vision, 24(3):271–300, 1997.
  • [5] Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE transactions on pattern analysis and machine intelligence, 29(6):1052–1067, 2007.
  • [6] Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence, 40(3):611–625, 2017.
  • [7] Jakob Engel, Thomas Schöps, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In European conference on computer vision, pages 834–849. Springer, 2014.
  • [8] Jakob Engel, Jurgen Sturm, and Daniel Cremers. Semi-dense visual odometry for a monocular camera. In Proceedings of the IEEE international conference on computer vision, pages 1449–1456, 2013.
  • [9] M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
  • [10] Christian Forster, Matia Pizzoli, and Davide Scaramuzza. Svo: Fast semi-direct monocular visual odometry. In 2014 IEEE international conference on robotics and automation (ICRA), pages 15–22. IEEE, 2014.
  • [11] A. Geiger, P. Lenz, C. Stiller, and R. Urtasun. Vision meets robotics: The kitti dataset. The International Journal of Robotics Research, 32(11):1231–1237, 2013.
  • [12] R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge Univ Pr, 2003.
  • [13] R. Hartley and A. Zisserman. Multiple View Geometry in computer vision, 2nd Edition. Cambridge Press, 2003.
  • [14] Peter J Huber. Robust statistical procedures, volume 68. Siam, 1996.
  • [15] Kenichi Kanatani. Geometric information criterion for model selection. International Journal of Computer Vision, 26(3):171–189, 1998.
  • [16] Georg Klein and David Murray. Parallel tracking and mapping for small ar workspaces. In Mixed and Augmented Reality, 2007. ISMAR 2007. 6th IEEE and ACM International Symposium on, pages 225–234. IEEE, 2007.
  • [17] D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004.
  • [18] Y. Lu and D. Song. Visual navigation using heterogeneous landmarks and unsupervised geometric constraints. In IEEE Transactions on Robotics (T-RO), volume 31, pages 736 –– 749, June 2015.
  • [19] R. Mur-Artal and J. D. Tardós. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Transactions on Robotics, 33(5):1255–1262, 2017.
  • [20] David Nistér. An efficient solution to the five-point relative pose problem. IEEE transactions on pattern analysis and machine intelligence, 26(6):0756–777, 2004.
  • [21] Théodore Papadopoulo and Manolis IA Lourakis.

    Estimating the jacobian of the singular value decomposition: Theory and applications.

    In European Conference on Computer Vision, pages 554–570. Springer, 2000.
  • [22] Marc Pollefeys, Frank Verbiest, and Luc Van Gool. Surviving dominant planes in uncalibrated structure and motion recovery. In European conference on computer vision, pages 837–851. Springer, 2002.
  • [23] Rahul Raguram, Ondrej Chum, Marc Pollefeys, Jiri Matas, and Jan-Michael Frahm. Usac: a universal framework for random sample consensus. IEEE transactions on pattern analysis and machine intelligence, 35(8):2022–2038, 2012.
  • [24] Rahul Raguram, Jan-Michael Frahm, and Marc Pollefeys. Exploiting uncertainty in random sample consensus. In 2009 IEEE 12th International Conference on Computer Vision, pages 2074–2081. IEEE, 2009.
  • [25] Peter J Rousseeuw. Least median of squares regression. Journal of the American statistical association, 79(388):871–880, 1984.
  • [26] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski. Orb: An efficient alternative to sift or surf. In 2011 International Conference on Computer Vision, pages 2564–2571. IEEE, 2011.
  • [27] Davide Scaramuzza. 1-point-ransac structure from motion for vehicle-mounted cameras by exploiting non-holonomic constraints. International journal of computer vision, 95(1):74–85, 2011.
  • [28] Henrik Stewenius, Christopher Engels, and David Nistér. Recent developments on direct relative orientation. ISPRS Journal of Photogrammetry and Remote Sensing, 60(4):284–294, 2006.
  • [29] Philip HS Torr. Geometric motion segmentation and model selection. Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 356(1740):1321–1340, 1998.
  • [30] Philip HS Torr, Andrew W Fitzgibbon, and Andrew Zisserman. The problem of degeneracy in structure and motion recovery from uncalibrated image sequences. International Journal of Computer Vision, 32(1):27–44, 1999.
  • [31] Philip HS Torr and Andrew Zisserman. Mlesac: A new robust estimator with application to estimating image geometry. Computer vision and image understanding, 78(1):138–156, 2000.
  • [32] Philip HS Torr, Andrew Zisserman, and Stephen J Maybank. Robust detection of degenerate configurations while estimating the fundamental matrix. Computer vision and image understanding, 71(3):312–333, 1998.
  • [33] Sebastiano Vigna. An experimental exploration of marsaglia’s xorshift generators, scrambled. ACM Transactions on Mathematical Software (TOMS), 42(4):30, 2016.