1 Introduction
Ellipses are quite common in natural or artificial scenes. The detection of ellipses in a fast and reliable manner from real world images provides a powerful analysis tool for many computer vision applications such as wheels detection
[1], biological cell division [2], and object segmentation for industrial applications [3]. Ellipse detection still remains unresolved as one of the classical tasks with long history. Most existing methods perform far from real time, which hinder their applications in reality.The earliest ellipse detection algorithm dates back to the classical Hough transform (HT) that fits the parametric form of an ellipse using a voting scheme [4]. The standard HT approach extracts ellipses by finding the clusters in a fivedimensional (5D) parametric space, consuming a great deal of memory and time. The randomized HT (RHT) improves the performance by reducing the number of false alarms [5]. The iterative RHT (IRHT) speeds up RHT so significant by focusing on the candidates likely to be an ellipse that it only needs 1D accumulators [6]. However, both RHT and IRHT are still quite slow attributing to the voting processing among numerous candidates, and the geometry relationships between points are also neglected during voting.
Researchers introduce algebraic or geometric constraints on points of an ellipse to screening candidates. Liang et al. [7]
introduce the maximum correntropy criterion into the constrained leastsquare fitting to alleviate the influence of outliers. Mulleti
et al. [8] use the finite rate of innovation sampling principle to fit noisy or partial ellipse. Both methods produce ellipses with less bias. However, they can only work on the image with one single ellipse. Xie and Ji exploit the symmetry of two points on ellipse, reducing the voting parameter to one [9]. However, it is timeprohibitive to enumerate every point pairs as elliptical candidates. Basca et al. [10] accelerate Xie et al.’s method with RHT by considering only a small random subset of initial point pairs. Zhang and Liu [11] use edge directional properties to reduce point combinations that lie out of the same ellipse boundary.Many methods take into account the geometric constraints on arc segments as the symmetry between points brings too many candidate pairs. Kim et al. [12] extract arc segments approximated by short straight lines. Libuda et al. [13] and Prasad et al. [14] improve Kim et al.’s method with less memory usage. However, these candidate arcs connecting short line segments may merge intersected arcs from different ellipses, resulting in lower precisions [12, 13]. Nguyen et al. [15] detect ellipses upon arcs by edge grouping. Their method is able to handle incomplete ellipses, but fails to detect ellipses splitting into many short arcs. Some other works formulate the mergence of elliptical arcs as an assignment problem, and iteratively correct the detection results [16, 17, 18]. These methods have high detection rates, but suffer from heavy computational costs. Prasad et al. [19] merge elliptical arcs with the relationship score given by the center of the ellipse fitting the arcs. Recently, Fornaciari et al. [20]
develop an ellipse detector that classifies elliptical arcs into four groups and estimates the ellipse parameters using the decomposed parameter space. There are still a number of candidate arcs in each group while their method renders a relatively faster detection than previous methods. Especially, it is quite time consuming to calculate every possible combinations of arcs from each group, not to mention that many of them are wrong combinations.
All aforementioned methods start the estimation from points or arcs with their positional constraints. Actually, there exist more constraints whether points or arcs are amenable to the analytical ellipse equation. In [21], RANSAC is used to randomly choose five points repeatedly until the ellipse determined by these five points closely passes through a maximum number of edge points. This method is inefficient as there exist so many combinations of five points and one has to calculate ellipse parameters for each fivepoint combination. Instead of directly calculating the ellipse parameters, Liu and Hu [22] use geometric distances of points to a conic in order to evaluate the similarity between any two of selected arcs. There are still a large number of wrong arcs combinations, and the computation of distances between points and arcs lower down its efficiency.
In this paper, we circumvent the high computational load by pruning and picking candidates using a projective invariant, named characteristic number (CN) [23]. The projective invariance of CN is introduced in [24] acting as geometric constraints for fiducial point localization under face pose changes. Later, Jia et al. employ this invariant property to construct a shape descriptor robust to perspective deformations [25]. For the first time, we explicitly take the advantage of the CN property giving the characterization of the intrinsic geometry of an underlying planar curve of points. The invariant gives the value of on any three collinear points and for any six points on a conic (ellipse or other conic curves). We are able to efficiently pruning straight lines and to determine whether noncollinear six points belong to the same conic without need of analytically fitting nor voting. The calculation of CN is also quite fast since the determinants of matrices constitute the primitives of CN.
Figure 1 illustrates the pruning and picking on an input image with a bunches of ellipses in Fig. 1. Candidate arc segments appear different colors, and we are able to prune those line segments having the CN value . We use six points on two arc segments, three on each, to calculate a CN value. The points on the bold arcs in pink and blue form the light brown triangle, while those on blue and green bold arcs generate the light green triangle. These triangles (closed loops) formed by auxiliary lines connecting arc points are necessary for CN calculation. The six points on the arc pairs from the same ellipse, e.g., the blue and green pair in Fig. 1, give the CN value . Thus, the arcs belonging to one ellipse easily pop out by CN values, and Fig. 1 shows the ellipse candidates formed by these picked arc segments. Figure 1 presents the final detection results by further ellipse fitting and validation. The proposed method is robust with high efficiency, suitable for realtime applications as validated in experiments.
The rest of this paper is organized as follows. Section 2 introduces the characteristic number (CN) and its property on lines and conic curves. Section 3 elaborates our fast ellipse detection algorithm with line pruning and arc picking using CN. Section 4 demonstrates experimental evaluations on accuracy and efficiency. Section 5 concludes the paper.
2 Characteristic number on line and conic
In this section, we first introduce the general definition of the characteristic number (CN), and give its computation on three collinear points (CNL) and six points in a conic curve (CNC), respectively. Also, we present the CN properties for these two configurations, the key to efficiently pruning line segments and picking candidates for elliptical arcs.
2.1 Characteristic number
The characteristic number extends the classical cross ratio in various respects, and reflects the intrinsic geometry underlying given points. The CN value of three collinear points is , while six points lying on a conic curve have a common CN value . We give the definition of below [23].
Definition 1 Let be dimension projective space over , and be distinct points in that construct a closed loop (). There are distinct points on the line segment Each point can be linearly represented by and as
(1) 
Let and , the quantity
(2) 
is defined as the characteristic number of and .
The classical cross ratio is a special case of CN when and . The characteristic number is proven to be a projective invariant, meaning that CN values keep unchanged under projective/perspective transformations.
The definition of CN requires a closed loop with sides, and the number of points at every side has to be identical. Subsequently, we draw auxiliary lines to set up this geometric configuration required for calculating CN on three collinear points and six points on a conic curve (ellipse).
2.2 Characteristic number on three collinear points
Three pairwise intersecting lines , and intersect at three points , and , forming a closed loop. These three lines intersect another line at three points , and , as shown in Fig. 2. According to Definition 1, , and can be linearly represented by , and as:
(3) 
Substituting the coefficients into (2), we calculate CN on three collinear points (CNL) as (4). It can be proved that the CN value equals to if , and are collinear [24].
(4) 
The proof process also implies that the calculation of CNL is achieved by the determinant of a matrix. In this study, we use this property to prune line segments in ellipse detection.
2.3 Characteristic number on six points of a conic
We denote the line through two points and as , and the intersection of two lines as . Let be six distinct points on a conic (ellipse) as shown in Fig. 3, and , and be three intersection points of the lines connecting some pairs of the six points on the conic:
(5) 
Similar to (3), each point of can be linearly represented by a pair of points from .
(6) 
We have the characteristic number on six points of a conic (CNC) as (7) by substituting the representation coefficients and into (2), and the CNC value equals . We apply this property of CN to pick arc segments likely lying on an ellipse.
(7) 
We provide a simple proof to based on Pascal’s hexagon theorem [26] as below.
Proof Let be six points on a conic. As shown in Fig. 3, we can obtain three more intersections as
(8) 
Then , , and can be represented by and through simple calculations as
(9) 
where we use homogeneous coordination to represent a planar point as , and denotes the determinant of the matrix given by the homogeneous coordinates of the three points , and as
(10) 
As the homogeneous coordinate of a point in a projective plane is independent on the initial points constructing the plane, we specify , and . Consequently, the points , and can be represented by , and by substituting (6) into (9) as
(11) 
3 Fast ellipse detection
In this section, we present our ellipse detector using the characteristic number to prune nonelliptical line segments and pick arc segments lying on an ellipse. These pruning and picking processes significantly reduce the number of arc candidates for final fitting, rendering fast detection. The complete detection procedure includes preprocessing, line pruning, arc selection, and parameter fitting and ellipse validation.
At the preprocessing step, edge points are detected and linked to generate arc segments, where short segments are removed as noise. We delete those arc segments likely to be lines detected by CNL values at the line pruning step. This step reduces the number of arc segments that are not parts of any ellipses. However, the possible arc combinations are still too many to efficiently fit elliptical parameters. At the following arc selection step, we firstly divide arc segments into four groups, and remove some impossible arc combinations across these groups according to their relatively positional relationships. Subsequently, we apply CNC to pick the arc combinations belonging to one ellipse. Only those picked arc segments are used to fit elliptic parameters in the last fitting and validation step. The pruning and picking with CNL and CNC significantly reduce the number of possible arc combinations that determine the computational load of the fitting. Also, these pruning and picking steps run fast so that the overall detector is quite efficient.
3.1 Preprocessing
Given an image, we firstly smooth the image to partially suppress noise, and apply the Canny edge detector [27] with default thresholds in Matlab to extract consecutive edge points. The edge detector outputs both the position and , and gradient on each edge point as , where , = , is the number of edge points.
It is possible to apply the CN constraint on any three or six edge points to determine whether they lie in a line or ellipse. However, we have to calculate and point combinations for edge points, resulting in high computational complexity. Moreover, most of these combinations come from noise, or different ellipses and lines, spending a large amount time on invalid CN calculations. Instead, we apply the constraints on arc segments.
In order to efficiently shear invalid arc combinations for later processing, we group arc segments into four sets corresponding to the arcs from one ellipse distributing in four quadrants as , , , and , named quadrant sets. In the preprocessing step, we separate edge points into two groups and by the signs of edge gradients as the first stage of this arc grouping:
(13) 
We link each edge point with the other edge points in its eight neighborhood from the same group with a breadthfirst strategy until no edge point exists in the neighborhoods. Consequently, we separately generate a series of arc segments in two groups shown in Fig. 4. Figure 4 demonstrates the arcs by linking edge points from the Canny detector (shown in Fig. 4) of the group , and Fig. 4 shows the arc segments of .
As the second stage of arc grouping, we divide each group into two sets, eventually producing four sets. As shown in Fig. 5, the vertices , , , and constitute the bounding box of an arc with the length of , where the starting and ending edge points of the arc are and , respectively. Denoting the difference between the numbers of pixels above and below (slashed and solid white blocks in Fig. 5) as , we split each group into two by the sign of . Taking an arc segment in the group for example, the arc falls in the set if the pixel number above the arc is greater than the below () otherwise () falls in the set. The last plot in Fig. 5 illustrates this splitting processing. We preform this stage of grouping after line pruning, and hence the cases of are quite rare. Finally, every arc segment in each of these four sets comes from one unique ellipse so that only arc combinations from different sets are necessary for the arc picking process. Therefore, this preprocessing is also designated for an efficient ellipse detector.
3.2 Line pruning
As a typical treatment to noise in literature, we simply set a threshold ( in all experiments of this study) to remove short segments. Following the noise attenuation, we employ CNL to prune line segments unlikely to come from an ellipse. The determinant of three edge points, the starting, midpoint, and ending of an arc segment ( and ) is taken as the fast calculation of the CNL value for the segment given in (12). The geometric interpretation of the determinant is the area of . We use the ratio of the area to the length of the arc () to robustly estimate whether these three points are collinear. The ratio close to zero indicates a line segment to be removed. Figures 4 and 6 presents the segments after removing noise and lines. The top row of Fig. 6 gives the arc segments with edge linking, and the bottom provides those after noise and line pruning. Many segments from noise and lines disappear, significantly reducing the number of arc candidates.
3.3 Arc selection
In this step, we pick candidate arcs, that are likely to assemble an ellipse, across the four sets where noise and line segments have been removed. Specifically, two arcs, each with three edge points, are taken from two different sets (one arc per set) in order to construct CNC. As shown in Fig. 7, there are two arcs and , where , and are the mid and two endpoints of , respectively. Similarly, , and are the points in . We intersect and at , and at , and and at . Given these points in and , we are able to have the representation coefficients and ( and ), and calculate the CNC value for this arc combination by substituting these coefficients into (7). Equation 7 also tells that the CNC value equals if these two arc segments with six points come from one identical ellipse. Therefore, picking two arc candidates from one ellipse turns out to simply comparing the CNC value of the arc combination with the value . Similar to line pruning, we can also use a threshold () that determines false negative for this process. The arc pair belonging the same ellipse obtains the value of CNC close to , and the absolute difference of CNC value and , represented as , is close to 0, as shown in Fig. 8 where the triangle constructed by the six points from the pair is labeled in light green. The redbrown triangle indicates the arc pair having the CNC value greater than , where the two arc segments of this pair lie on different ellipses.
We only consider the combinations with arc segments from two adjacent quadrant sets, e.g., and , but leave out those twoarc combinations from diagonal sets and . This choice further lowers down possible combinations, and more importantly improves the robustness to calculate CNC values. As shown in Fig. 7, the lines and are likely to be parallel for arcs from diagonal quadrant sets so that the intersection does not exit, or lies quite far from the edge points. The calculation of CNC value is so instable that the determination of ellipse is no more effective. We exclude the combinations with arcs from two diagonal quadrant sets, avoiding this instability.
The combinations with more arc segments are also able to yield more accurate parameter estimation in the later step. Unfortunately, noise and/or occlusions are likely to bring the absence of arc segments from one quadrant set. Hence, we constitute the combinations with three arcs from three different quadrant sets upon the arc pairs picked by CNC for later parameter fitting. There are four kinds of valid arc combinations from three quadrant sets for an ellipse, as shown in the first row of Tab. I. We pick out these combinations in each kind by using both coordinate and CNC constraints together, where the coordinate constraint takes relative locations of arcs to remove invalidate combinations. The second and third rows of Tab. I list the picking rules for valid combinations using CNC and coordinate constraints, respectively. The symbols and denote the starting and end points of . Figure 9 illustrates two examples of invalid threearc combinations that form one ellipse, removed by the constraints in Tab. I.
Valid arc combinations  

Sets constraints  
Coordinate constraints 
Specifically, the picking process for a threearc combination begins with an arc segment in the middle quadrant set, and then proceed to those in the other two sets. We find an arc pair first, and then the third arc to form the combination by alternatively applying the coordinate and CNC constraints. We take the set combination in Tab. I as an example to illustrate the picking process. We starts with one arc in , and test the pair of the arc of and every arc segments in with the coordinate constraint. If one pair meet the coordinate constraint, the CNC constraint is applied to this pair further. Subsequently, we search the set to find the third arc segments forming a pair with the arc in that follow the coordinate constraint. Herein, the CNC constraint runs only once in order to balance the speed and accuracy. We repeatedly run the picking process for every arc segments in , and find all valid threearc combinations for . Algorithm 1 details the picking process for this set combination. The similar process applies to the other threeset combinations in the first row of Tab. I, and finally we have all valid threearc combinations ready for parameter fitting given in the next section.
3.4 Parameter fitting
There are five parameters to determine an ellipse, including its center (two), orientation (one), and major and minor semiaxes (two). We follow the procedure in [20] to estimate all these parameters but the center whose calculation we provide below.
We estimate the center as the intersection of auxiliary lines generated from any twopoint pairs on the three arcs picked by the previous procedure. As illustrated in the top figure of Fig. 10, and are two points on an arc segment, is the intersection of the two tangent lines from and , and is the middle point of the line segment . It is proved that the line passes through the ellipse center . For practical images, it is unnecessary that all such auxiliary line segments like given by any point pairs on the elliptical arcs, but we are able to locate the center as the point closest to those lines by the least square fitting. In this study, we find that the accuracy of this fitting (estimation) also depends on the tangents of the points along the ellipse.
We use ( for our experiments) parallel chords instead of tangent lines in order to minimize the effects from tangent deviations, as shown in the bottom figure of Fig. 10. The points and are the mid points of the arcs and , respectively. We generate chords parallel to , and so we do to the chord . The points and () are the midpoints of the two series of parallel chords to and , respectively. The points of are collinear at , lying on the line , and the intersection of and determines the ellipse center . We estimate and using a fast variant of the robust TheilSen estimator [28] with two arcs in adjacent quadrant sets. Consequently, we obtain four lines through the ellipse center generated from a threearc combination, yielding at most six pairwise intersections. The algebraic average of the coordinates of these six intersections is taken as the output ellipse center .
3.5 Ellipses validation
Candidate ellipses are available after the parameter fitting step. There may exist false positives or duplicated ones in these candidates so that a validate step is necessary.
We provide two indices to validate the candidates. The first one measures how many edge points fitting the corresponding ellipse. The more edge points fit, the more likely the ellipse exists. We substitute every edge points into the correspondence ellipse equation to calculate how it fits the equation. We count the number of the fitting edge points, and calculate the ratio to the total points that generate the ellipse by fitting arc combinations.
The second index accounts for arc lengths of threearc combinations giving ellipse parameters. Ideally, the length of an arc combination is larger than the sum of two semiaxes. We use the ratio between the total length of one threearc combination and three times of the semiaxes summation as the index. Larger values indicate more stable detection of the ellipse.
The validation in [19] is able to discover duplicated ellipses, which allows to assess the similarity of two ellipses by comparing the differences of ellipse parameters. A voting strategy is adopted to pick up the center of a given cluster, removing duplicate ellipses.
4 Experimental results and analysis
We perform a series of experiments on data sets with both synthetic and real images to evaluate the performance of our fast detector by comparing with other stateoftheart methods ^{1}^{1}1The source code and resultant images of our detector can be found at https://github.com/dlutdimt/ellipsedetector.
4.1 Evaluation metrics and data sets
All the experiments in this paper are executed on a desktop with Intel(R) Core(TM) i76700 CPU whose clock frequency is GHz. The performance of ellipse detectors is evaluated in terms of running time and Fmeasure. The most timeconsuming step in many existing methods lie in parameter fitting because too many arc or point candidates are fed to the step. In addition to executing time, we count candidate combinations of arcs (CC) for ellipse fitting, and use CC as a direct index to demonstrate the impacts of CNC and CNL.
Fmeasure is defined as:
(14) 
where
(15) 
The symbol denotes the number of detected ellipses, and indicates the number of groundtruth ellipses. is the number of correctly detected ellipses. The overlapping ratio of a detected ellipse to the ground truth is defined as:
(16) 
where is the number of pixels inside the ellipse . The detected ellipse is considered as a correct detection if . The threshold is set to 0.8 throughout our experiments, as did in [29].
Experimental data contain both real world and synthetic images. We use data sets with real world images to validate efficiency and accuracy, while synthetic ones are designated to demonstrate robustness to noise and ellipse variations. Real world images are those from Dataset #1 and Dataset Prasad, the same as [20], for fair comparisons. Dataset #1 is composed of 400 images having elliptic shapes, collected from MIRFlickr and LabelMe repositories [30]. Those MIRFlickr images are of high quality, and most of them contain only one object (ellipse), while those from LabelM are noisy images of low resolution, containing multiple objects. Dataset Prasad contains 198 real images from [29], where objects of oval shapes like human faces are regarded as ellipses. Besides evaluations on the original Dataset Prasad, we also construct Dataset #2 by selecting 50 images with rigorous ellipses in order to show the accuracy of our method on detecting ellipses. These rigorously elliptical shapes are quite common in industrial inspection and diagnosis.
4.2 Performance analysis
The effectiveness of line pruning and arc selection steps are illustrated by comparisons on detection results with and without these steps. We also provide empirical studies on the hyperparameters involved in these steps. The parameters for ellipse fitting and validation are taken the same values as those in [20].
4.2.1 Performance analysis of line pruning
We remove short and straight line segments using CNL as discussed in Section 3.2. Our goal is to prune noise effects and lines in input images as much as possible but preserve arcs of ellipses. We perform line pruning on Dataset #1 by tuning from to . Intuitively, the threshold limits the height of the triangle formed by the three edge points for CNL calculation. Therefore, the larger is, the more arcs are to be removed. The zero threshold, , indicates no line pruning step included.
Table II lists the values of arc numbers, running time, and Fmeasure with varying values. There are averagely arc segments without line pruning (), and more segments are removed when increasing values. The computing time is ms, and Fmeasure is without line pruning. When noise and line segments are removed by CNL, the computing time is becoming lower and Fmeasure increases. We obtain the best performance, ms for computation and for Fmeasure when . The computing time is still decreasing when more arc segments are pruned by the thresholds larger than . The values of Fmeasure slightly decrease due to increased false negative. These results show that the line pruning step using CNL is effective to remove noise and lines, and also alleviate the computational load for later steps. We set to balance the accuracy and computing time in all comparisons with the others.

arc num  avg. time(ms)  Fmeasure 

0  181  28.86  0.4313 
1  170  26.52  0.4338 
2  108  13.75  0.4576 
3  65  8.52  0.4692 
4  40  6.79  0.4499 
5  26  6.17  0.4226 
4.2.2 Performance analysis of arc selection
We use the geometric constraint of six points on arcs derived from CNC to pick up arc segments belonging to one ellipse. Theoretically, the CNC value of six points lying on an ellipse equals , but various imaging conditions (e.g., thermal noise and lens distortions) in practical applications may cause the value deviating from . As discussed in Section 3.3, we relax this hard constraint to a range in the vicinity of determined by . Herein, we perform experimental analysis on the relationship between point coordinates and CNC values. This analysis does not only give arise to an appropriate threshold, but also validates the effectiveness of the arc selection based on CNC.
Supposing an ellipse centering at the coordinate origin, we fix five distinct points on the ellipse, and vary the sixth point around the ellipse to show the distribution of CNC values in Fig. 11. Different colors indicate various CNC values given by the color bar on the right most of Fig. 11. All the CNC values higher than are colored in redbrown, while all those lower than in blue. Figure 11 illustrates that most of CNC values with the sixth point close to the ellipse fall within the range from to . There exist several regions where CNC values lie in the range while the sixth point locates far from the ellipse. These regions include the starshaped area out of the ellipse between and , and the bottom left and top right ends of the line stretching out of . This observation can be explained by the fact that the CNC value of indicates the points lying on a conic curve not only an ellipse. However, these ’outliers’ have few effects on our detector, because other types of conic curves, e.g., hyperbola and parabola, seldom appear in practical industrial images, and the majority can be also removed by the coordinate constraint even if a few of them appear.
We further apply our detector on dataset #1 by varying , the absolute deviation from , from to , giving seventeen values in total. Figure 12 illustrates the values of computing time with varying thresholds. The blue line in Fig. 12 demonstrates the seventeen values of in an ascending order, also listed in the third row below the plot. The step values vary with the value ranges of , i.e., for the range from to , for to , and for values larger than . The orange and grey bars show the computing time with and without the arc selection step, whose values are listed in the first and second rows below, respectively. The average computing time is ms for each image, one half of the detection time without CNC, ms, when we use the hard constraint picking only a small fraction of arc segments. More arc segments are included for parameter fitting, demanding more computing time, when increasing the threshold values. When is larger than , the detector spends more time than that without the selection step since the calculation of CNC takes more time than what the constraint can save. As a result, the orange bars are higher than the grey ones for these values in Fig. 12.
Figure 13 shows the values of Fmeasure (Fm) with varying threshold values. The Fmeasure for the detector without the CNC constraint is labeled in the gray bar, while ours in orange bars. The hard constraint yields a very low Fm since the choice excludes many arc segments slightly deviating from an ellipse, resulting in significant false negatives. The threshold outputs the best Fm , even higher than the detector without this constraint. This improvement shows that the CNC constraint is also able to exclude false positives in addition to decreasing the computing time from ms to ms. As expected, the values of Fm given by the thresholds larger than are quite close to that without the selection step. In these cases, the selection step takes no effect on lowering down false positives. Therefore, the arc selection step using CNC is quite crucial to both efficiency and accuracy. We choose the threshold as to generate the best performance in the following experiments.
4.3 Comparisons with the state of the art
Firstly, we compare our detector with three recent arcbased methods, i.e., Zhang [11], Libuda [13], and Fornaciari [20], on Dataset #1. The set consists of images with different qualities and various numbers of target ellipses. The latest works of [7] and [8] are not so relevant as these three because both are pointbased and applicable to sceneries where only one ellipse appear. The execution program of [20] is provided by the authors, and we take the results of [11] and [13] reported in [20]. Table III lists the comparisons on average running time and Fmeasure. The method of Zhang [11] performs the worst: the fitting on a large number of pixel combinations takes much time over s for an image; its Fmeasure is the lowest as . Libuda [13] and Fornaciari [20] output the similar efficiency with the average execution time for each image ms and ms, respectively. The values of their Fmeasure are also quite close. Our detector achieves the best in terms of both efficiency(ms) and accuracy (Fmeasure about ). Specifically, we buy 7% higher accuracy with 33% less execution time than the fastest method [20].
Method  Fmeasure  avg.time(ms) 

Libuda[13]  0.4258  14.38 
Prasad[29]  0.4512  823.38 
Zhang[11]  0.3421  4243.86 
Fornaciari[20]  0.4385  12.79 
Our  0.4692  8.54 
Several examples of the above methods on Dataset #1 are shown in Fig. 14 where the first column is the input images, and the second one lists the ground truth (GT). False negatives occur in all the results of Libuda [13] except for the fifth row. The method of Fornaciari [20] works quite close to ours that correctly detects almost all the ellipses in these images. It is worth noting that only our method successfully picks out the middle wheel partially occluded by the lady in the third image, while the other three fail. Compared with the ground truth, our detector outputs one extra ellipse out of the bottom right tray. Actually, one can find a dim elliptical trail along the tray, but the ground truth neglects it.
Secondly, we compare the execution time on each processing step with thestateoftheart on Dataset #1, as shown in Tab. IV. The method of Zhang [11] spends about ms on estimating the parameters due to huge numbers of possible point combinations. The method of Prasad [29] reduces the time on parameter estimation via grouping arcs with curvature and convexity, but the grouping spends additional ms. The method of Libuda [13] uses an iterative strategy, and the time spent on each of the first three steps equals about ms. Fornaciari’s detector [20], the fastest one among the existing, spends the most time ms on grouping arcs with their relative locations against the other steps. In contrast, our method only uses about half of the execution time of [20] on the grouping step, and the total execution time is less than two third of [20]. Thus, our detector reduces the time for the grouping step, the bottleneck for efficient detectors.
Libuda [13]  Prasad [29]  Zhang [11]  Fornaciari [20]  Our  

Edage detection  4.49  3.54  3.97  3.45  3.43 
Preprocessing  4.15  78.03  3.55  1.94  1.90 
Grouping  4.89  278.01  0.25  4.90  2.53 
Estimation  0.84  3.40  4236.06  2.30  0.63 
Validation and Clustering  0.00  460.39  0.03  0.21  0.06 
Total  14.38  823.38  4243.86  12.79  8.54 
Further, we peer into the proposed detector to analyze how line pruning and arc selection accelerate detection. Table V illustrates the effects of CNL and CNC by listing the averaged values of arc numbers, arc combination numbers (CC) for parameter fitting, execution time and Fmeasure on Dataset #1. Without any processing using CNL nor CNC, possible averaged arcs and arc combinations (CC) are and , respectively. After line pruning, the arc number reduces to , about one third of of the original, and the CC value significantly decreases to 196, of the original. Only arc combinations remain for parameter fitting, shearing combinations from the original. Naturally, the execution time drops down from ms to ms. The value of Fmeasure also increases from to since many false positives are removed by these two steps using CNL and CNC. Green curves in the first and second rows of Fig. 15 present possible ellipses to be fitted without and with our pruning/selection steps, respectively. It is evident that the green curves in the second row are much less than those in the first row, intuitively showing the effectiveness of our CNL/CNC based processing to save time and improve accuracy.
After arc detection  After line pruning  After arc selection  

arc Num  181  65  65 
CC  2619  196  40 
avg.time (ms)  129.33  12.73  8.52 
Fmeasure  0.3163  0.4090  0.4692 
Finally, we compare our detector with [20], giving the best performance among the existing, in terms of CC, Fmeasure, and execution time on three data sets including Datast #1, Dataset #2, and Dataset Prasad. As shown in Tab. VI, our method outperforms [20] on all the three data sets. The last column shows the improvement percentage both on efficiency and accuracy. The accuracy of the proposed detector increases about 6.6% in average over [20], and more significantly our detector averagely runs about 31.0% faster than [20] on all the three data sets. These results show that our detector is applicable to real time scenarios in practice.
DataBase  avg. time(ms)  Fm  CC  improvement percentage  

[20]  Ours  [20]  Ours  [20]  Ours  speed  accuracy  
Dataset Prasad  4.34  3.35  0.2874  0.3059  82  19  22.70%  6.44% 
Dataset #1  13.58  8.55  0.4385  0.4692  267  40  37.05%  7.00% 
Dataset #2  5.26  4.09  0.5893  0.6271  117  34  22.27%  6.41% 
4.4 Robustness to ellipse variations and noise
In order to investigate the robustness to ellipse variations and noise, we use two synthetic data sets with different orientations and ratios of two semiaxes of ellipses, and also apply saltandpepper noise to real world images. The noise break arcs into several small fragments, which may affect the accuracy and efficiency.
The first synthetic data set consists of ellipses with various semiaxes ratios and orientations. One semiaxis is fixed as , and the other one varies so that the ratios range from to at the step of . Orientations vary from to at the step of . The second data set contains images with a fixed center and orientation, showing changes on ratios and lengths of semiaxes. One semiaxis varies from to at the step of , and the other one changes accordingly so that axes ratios range from to at the step of . Both data sets come from [20] that generates synthetic images, each containing one single ellipse without noise.
Figure 16 illustrates comparisons with [20], where black points indicate failures of detection. The results on the first data set are given in Fig. 16(a) and Fig. 16(b). The horizontal axis gives ratios of two semiaxes, and the vertical one shows orientations. Both methods are robust to orientation changes as long as axes ratios are larger than 0.25, but they fail in the cases of small axes ratios when ellipses degenerate into straight lines. The results on the second data set are shown in Fig. 16(c) and Fig. 16(d), where the vertical axis indicates lengths of major axes. The robustness of both methods is quite similar, working well on ellipses whose major axis is longer than and axes ratio is larger than . Small arcs are likely to be pruned as noise when the ellipse only has a few pixels. Fortunately, one may tackle the problem of small ellipses by upscaling the image as did in [20]. Therefore, the proposed method is quite robust, and only fails in some extreme cases, e.g., small and extremely oblate ellipses, which are quite rare in reality.
We add saltandpepper noise to images of Dataset #2 with the ratios of noise to image set as , , , , and to validate the robustness of our method to noise. Again, we compare with [20] on efficiency and accuracy, shown in Fig. 17. The values of Fmeasure for these two methods on images without noise are and , respectively. The values decrease for both methods when the noise increases. When the ratio of noise to image is up to 18%, our value is about , higher than that of [20]. When the noise level is getting higher, the execution time of our detector stays around ms, and that of [20] fluctuates a bit around ms not so stable as ours. Generally, both methods runs faster as noise increases because they are able to remove small arc segments caused by noise. As a summary, noise has few effect on the efficiency, but slightly lower down the accuracy.
5 Conclusions
In this paper, an ellipse detector for realtime application is proposed. We trades off accuracy and efficiency and pay more attention to the execution time. We introduce a new geometry constraint to prune lines and select arcs belong to the same ellipse. The detector removes the straight arcs based on characteristic on line (CNL), and selects candidate elliptical arc combinations by characteristic on conic (CNC). Our method outperforms thestateoftheart by the experiments on real images, which can be used in realtime for various applications. In the future, we will improve our method to make it possible to detect quite small ellipses in images, which is a challenge for most existing methods.
Appendix A Proof of Pascal’s hexagon theorem
Pascal’s hexagon theorem: Let be different points on a nondegenerative conic , as shown in Fig. 3. Then three intersections
(17) 
are collinear.
Proof: The coordinate of , and can be represented by as
(18) 
where denotes the cross product of two points. Then , and can be represented through simple calculations as
(19) 
To prove that , and are collinear is equivalent to prove
(20) 
Then we can substitute (19) into (20), and get the equivalent equation as
(21)  
In order to prove it, we can replace any point with the general point , taking for example, then we can get the parametric equations of conic by other five points. As is one point on conic , it must meet (21). The proof is completed.
Acknowledgments
We thank Dr. M. Fornaciari, Dr. S. Mulleti and Dr. T. Lu for providing their executables and insights. We also thank Dr. M. Fornaciari for his experimental data.
References
 [1] T. Cooke, “A fast automatic ellipse detector,” in 2010 International Conference on Digital Image Computing: Techniques and Applications. IEEE, 2010, pp. 575–580.
 [2] S. Zafari, T. Eerola, J. Sampo, H. Kalviainen, and H. Haario, “Segmentation of overlapping elliptical objects in silhouette images,” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5942–5952, 2015.
 [3] C. Teutsch, D. Berndt, E. Trostmann, and M. Weber, “Realtime detection of elliptic shapes for automated object recognition and object tracking,” in Electronic Imaging 2006. International Society for Optics and Photonics, 2006, pp. 60 700J–60 700J.
 [4] J. Illingworth and J. Kittler, “A survey of the hough transform,” Computer vision, graphics, and image processing, vol. 44, no. 1, pp. 87–116, 1988.
 [5] R. A. McLaughlin, “Randomized hough transform: improved ellipse detection with comparison,” Pattern Recognition Letters, vol. 19, no. 3, pp. 299–305, 1998.
 [6] W. Lu and J. Tan, “Detection of incomplete ellipse in images with strong noise by iterative randomized hough transform (irht),” Pattern Recognition, vol. 41, no. 4, pp. 1268–1279, 2008.
 [7] J. Liang, Y. Wang, and X. Zeng, “Robust ellipse fitting via halfquadratic and semidefinite relaxation optimization,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 4276–4286, 2015.
 [8] S. Mulleti and C. S. Seelamantula, “Ellipse fitting using the finite rate of innovation sampling principle,” IEEE Transactions on Image Processing, vol. 25, no. 3, pp. 1451–1464, 2016.
 [9] Y. Xie and Q. Ji, “A new efficient ellipse detection method,” in 16th International Conference on Pattern Recognition, vol. 2. IEEE, 2002, pp. 957–960.
 [10] C. Basca, M. Talos, and R. Brad, “Randomized hough transform for ellipse detection with result clustering,” in EUROCON 2005The International Conference on ”Computer as a Tool”, vol. 2. IEEE, 2005, pp. 1397–1400.
 [11] S.C. Zhang and Z.Q. Liu, “A robust, realtime ellipse detector,” Pattern Recognition, vol. 38, no. 2, pp. 273–287, 2005.
 [12] E. Kim, M. Haseyama, and H. Kitajima, “Fast and robust ellipse extraction from complicated images,” in Proceedings of IEEE information technology and applications. Citeseer, 2002.
 [13] L. Libuda, I. Grothues, and K.F. Kraiss, “Ellipse detection in digital image data using geometric features,” in Advanc
Comments
There are no comments yet.