In order to mention the proposed method, some basic knowledge about quaternion algebra , quaternion Fourier transform [7, 14], quaternion Hardy space , Jin’s gradient operator  and Hough transform  are reviewed as follows.
1.1 Quaternion algebra and quaternion Fourier transform
denote the Hamiltonian skew field of quaternion
which is an associative anti-commutative four-dimensional algebra. A quaternion-valued function can be represented by
1.2 Quaternion Hardy space
Let , namely upper half complex plane and be the Hardy space on the upper half complex plane. Let . And a subset of is defined by The quaternion Hardy space consists of all functions satisfying [Hu1],
where , .
1.3 Jin’s gradient operator
Let be an color image that maps a point
to a vector, , . Specifically, for color image , let , , denotes the red, the green, and the blue channel, respectively. Then the square of the variation of at the position with the distance in the direction is given by
Then the gradient magnitude of the Jin’s gradient operator is given by
The gradient direction is defined as the value that maximizes over
where , While , is undefined.
1.4 Hough transform
The Hough transform is done on a binary image, obtained after processing the original image by an edge detector [25, 41, 35]. This is an ingenious method that converts such global curve detection problem into an efficient peak detection problem in parameter space. In addition, Dorst et al [11, 10, 9, 8] studied the application of plane-based geometric algebra in the representation of straight lines. The basic idea of Hough transform is to use the duality relationship between the points and lines in the image domain and the parameter domain. In particular, the slope-intercept form is a classical parametric form for straight line detection, as is shown in Eq. (1.11):
where is the coordinate of the pixel being mapped, and the parameters and are the slop and y-intercept of the line respectively. It’s worth noting that this method is sensitive to the choice of coordinate axes on the image plane because both the parameters become unbounded when is infinite.
Duda and Hart  lucidly resolved the issue of unboundedness by mapping a point in image space to a sinusoidal curve in parameter space. Here we use HoughlinesP method to detect lines, which is based on statistics not only could detect the two endpoints of the straight lines but also have the advantage of high efficiency. and satisfy the following relationship:
where is the angle between the line and the -axis, is the distance from the origin to the line in the image coordinate, and .
The first step of Hough transform is initialize accumulator to all zeros, followed by the parameter space is quantized in intervals of and and corresponding accumulators are created to collect the evidence (or vote) of object pixels satisfying Eq.(1.12). Consider the (, )th accumulator corresponding to intervals [(, ) and [(, ) where . For each object pixel satisfying Eq.(1.12) with parameters in the above range, the vote in the (, )th accumulator is incremented by 1. In the peak detection phase, the accumulators having number of votes above a critical threshold correspond to straight lines in the image. Therefore, the line detection problem is converted to peak statistics problems.
2 Quaternion Hardy filter
The analytic signal  of a given real signal is defined by
where denotes the Hilbert transform  of . By a direct computation, the Fourier transform of is given by
The analytic signal of can be regarded as the output signal of a filter with input . The system function of this filter is given by
Motivated by the success of analytic signal in the real signal processing, we try to use the alternative tool in the 2D signal processing.
Give a 2D quaternion-valued signal . The quaternion analytic signal  is defined by
where , are the partial Hilbert transforms with respect to respectively, while the 2D Hilbert transform. By a direct computation [Kou2], the quaternion Fourier transform of is written as
The quaternion analytic signal of can be regarded as the output signal of a filter with input . The system function of this filter is given by
By the inverse quaternion Fourier transform on the both side of (2.5), we obtain
In this paper, we apply quaternion Hardy filter as a lane detector of color image. The system function of quaternion Hardy filter is defined by
Based on the above analysis, we have the following theorem.
Let , and
Where is given by (2.8). Then .
The proof of Theorem 2.1 can be found in [QHF]. It states that the output signal of the quaternion Hardy filter belongs to the quaternion Hardy space .
In image processing, quaternion Hardy filter can not only suppress the high frequency of image, but also enhance the edge feature. It’s worth noting that, the system function plays a key role in quaternion Hardy filter. It has the advantage of both Hilbert transform and low-pass filter. On the one hand, the factor in performs Hilbert transform on the input signal. It can enhance the edge feature. On the other hand, could improve the ability of noise immunity for the quaternion Hardy filter.
3 The proposed lane detection algorithm
The flowchart is shown in Fig. 1. In the following, let’s consider the details of the proposed algorithm. They are divided by the following steps.
Step 1. Input the road image . By quaternion representation, it can be written as
where and represent the red, green and blue components of color image , respectively.
Step 2. Compute the discrete quaternion Fourier transform (DQFT) , , of the image from Step 1, we have
Step 3. For the fixed positive and , calculate by Eq. (2.8), we have
Multiplying with , from Step 2 and 3, respectively. By Theorem 2.1, we have
Step 4. Compute the inverse DQFT of , from Step 3,
We obtain .
Step 5. Extract the vector part of , we obtain
where , are real-valued functions.
Step 6. Applying the Jin’s gradient operator  to .
Step 7. Furthermore, perform the Hough operator .
Step 8. Finally, we show the final result by placing the resulting lane lines on the original image.
To verify the effectiveness of quaternion hardy filter-based lane detection algorithm, we conduct experiments on 27 video clips.
The experiments are programmed in Matlab R2016b.
The proposed algorithm is compared with previous work , , , , , , and . Here, Jin  means that the proposed method without steps 2-4 and provides a baseline reference results. The previous work involved six different approaches that are typical of the field of lane line detection.
The video clips used in the experiment were from public dataset , which were shot using dashboard cameras in the US and Korea. This data set contains a total of 49 video clips, and 27 are randomly selected for this experiment. The 27 video clips consist of 44159 frame images which divided into 6 different weather conditions: day-clear (5), day-rainy (5), day-snowy (5), night-clear (5), night-rainy (5) and night-snowy (2). The number in parentheses after the weather label represent the number of video clips used in the experiment.
4.1 Visual results
Visual results including rain night and snow night are illustrate in Fig. 2 and Fig. 3, respectively. When car lights form bright lines on the road on a rainy night, the comparison method may fail. It affects the detection of lane lines. For example, in the second row and fourth column of Fig. 2, Jin’s method is obviously affected by the light, thus making a misjudgment of the lane line. In addition, Jin’s method (e.g., the second row and first column in Fig. 3) detects the boundary line of the navigation screen and ignores the actual lane line on the right. Gradient-based approaches  may preserve the boundary portion of the navigation screen while ignoring the actual lane lines when they are reflective navigation screens are present. In general, the proposed method can achieve satisfactory results under complex weather conditions and illumination conditions. The proposed method cannot, however, detect lane markings when affected by surrounding buildings or windshield wipers, as shown in Fig. 4.
4.2 Quantitative comparisons
For each image, two boundary lines are drawn on the long edge of the lane mark in the presence of the lane mark, and the real value of the ground is obtained manually. The detection rate is obtained by the following
where represents the number of images correctly detected and represents the total number of images participated in the detection.
We present a quantitative evaluation for seven methods in Table 2. It can be seen from Table 2 that all the algorithms can generate good results when lane images are captured with good conditions, as shown in clear days and nights. For clear day sets, QHF-based method achieves a 99.4% accuracy among the compared methods. For rain day and snow day sets, the proposed method keeps steady in the accuracy. On clear night sets, the accuracy of the proposed method is still in the Top three. In the rainy night data set, the accuracy of the proposed method was still 88.5%, despite the fact that raindrops falling on the windshield caused considerable interference with the lane lines in the image. For the snowy night data set, the other six methods were not considered, and the proposed method was accurate to 95.4%. We have to say that there are some situations that are hard to dealt with. Fig. 4 gives several failure samples. For those images affected by wipers, illuminations and weather conditions, our method faces challenges. A potential solution to solve this problem is to use ROI (region of interest) . However, ROI-based lane detection method require much computation cost. Therefore, more optimization works needs to be done in the future.
|CPU||Unknow||Core 2||2.8GHz||Unknow||2.3GHz||A9 GHz||2.3GHz||2.3GHz|
5 Conclusion and future work
In this paper, we proposed a QHF-based lane detection method for color image. It exploits quaternion Hardy filter to generate clear feature image, and it use Jin’s gradient operator and Hough transform to provided lane lines, which ensures accuracy lane lines. Our method could handle with multiple situations such as various weather conditions, illuminations and lane markings. Experiments conducted on the dataset indicate that the proposed method achieves satisfactory results. The results were accurate and robust with respect to the complex environment lane markings. Finally, we are interested in extending our proposed quaternion lane line detection idea to color video for lane line detection in the future. In order to optimize the algorithm, we will also consider fast algorithms  and algorithm optimization software .
This research was supported by Macao Science and Technology Development Fund (No. FDCT/085/2018/A2).
-  Note: https://drive.google.com/file/d/1315Ry7isciL3nRvU5SCXM_-4meR2MyI/view?usp=sharing Cited by: §4.
-  (2008) Real time detection of lane markers in urban streets. In 2008 IEEE Intelligent Vehicles Symposium, pp. 7–12. Cited by: Table 2, §4.
-  (1981) Generalizing the hough transform to detect arbitrary shapes. Pattern recognition 13 (2), pp. 111–122. Cited by: §3, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2020) A robust color edge detection algorithm based on quaternion hardy filter. arXiv preprint arXiv:2001.01800. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2001) Hypercomplex signals-a novel extension of the analytic signal to the multidimensional case. IEEE Transactions on signal processing 49 (11), pp. 2844–2852. Cited by: §2.
-  (2018) Robust lane detection and tracking for real-time applications. IEEE Transactions on Intelligent Transportation Systems 19 (12), pp. 4043–4048. Cited by: §4.2, Table 2, §4.
-  (2019) Plancherel theorem and quaternion fourier transform for square integrable functions. complex variables and elliptic equations 64 (2), pp. 223–242. Cited by: §1.2, §1, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (1984) Spirograph theory: a framework for calculations on digitized straight lines. IEEE transactions on pattern analysis and machine intelligence (5), pp. 632–639. Cited by: §1.4.
-  (1984) Discrete representation of straight lines. IEEE Transactions on Pattern Analysis and Machine Intelligence (4), pp. 450–463. Cited by: §1.4.
Best linear unbiased estimators for properties of digitized straight lines. IEEE transactions on pattern analysis and machine intelligence (2), pp. 276–282. Cited by: §1.4.
-  A guided tour to the plane-based geometric algebra pga. Cited by: §1.4.
-  (1972) Use of the hough transformation to detect lines and curves in pictures. Communications of the ACM 15 (1), pp. 11–15. Cited by: §1.4.
-  (2014) Quaternion fourier transforms for signal and image processing. John Wiley & Sons. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (1992) Hypercomplex spectral transformations. Ph.D. Thesis, University of Minnesota. Cited by: §1, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  GAALOP. Note: http://www.gaalop.de/dhilden/ Cited by: §5.
-  (2007) Bounded analytic functions. Vol. 236, Springer Science & Business Media. Cited by: §2.
-  (2007) Quaternion fourier transform on quaternion fields and generalizations. Advances in Applied Clifford Algebras 17 (3), pp. 497–517. Cited by: §1.2.
-  (2010) Directional uncertainty principle for quaternion fourier transform. Advances in Applied Clifford Algebras 20 (2), pp. 271–284. Cited by: §1.2.
-  (2013) The orthogonal 2d planes split of quaternions and steerable quaternion fourier transformations. In Quaternion and Clifford Fourier transforms and wavelets, pp. 15–39. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2013) The orthogonal 2d planes split of quaternions and steerable quaternion fourier transformations. Quaternion and Clifford Fourier transforms and wavelets, pp. 15–39. Cited by: §5.
-  (2016) The quaternion domain fourier transform and its properties. Advances in Applied Clifford Algebras 26 (3), pp. 969–984. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2017) General two-sided quaternion fourier transform, convolution and mustard convolution. Advances in Applied Clifford Algebras 27 (1), pp. 381–395. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2017) Quaternion fourier and linear canonical inversion theorems. Mathematical Methods in the Applied Sciences 40 (7), pp. 2421–2440. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2018) Phase-based edge detection algorithms. Mathematical Methods in the Applied Sciences 41 (11), pp. 4148–4169. Cited by: §1, A Robust Lane Detection Associated with Quaternion Hardy Filter, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (1988) A survey of the hough transform. Computer vision, graphics, and image processing 44 (1), pp. 87–116. Cited by: §1.4.
Improved direction estimation for di zenzo’s multichannel image gradient operator. Pattern recognition 45 (12), pp. 4300–4311. Cited by: §1, §3, Table 2, §4, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2015) Efficient lane detection based on spatiotemporal images. IEEE Transactions on Intelligent Transportation Systems 17 (1), pp. 289–295. Cited by: Table 2, §4.
-  (2009) Hilbert transforms. Vol. 1, Cambridge University Press Cambridge. Cited by: §2.
-  (2017) Envelope detection using generalized analytic signal in 2d qlct domains. Multidimensional Systems and Signal Processing 28 (4), pp. 1343–1366. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2016) Uncertainty principles associated with quaternionic linear canonical transforms. Mathematical Methods in the Applied Sciences 39 (10), pp. 2722–2736. Cited by: §1, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2016) Nighttime lane markings recognition based on canny detection and hough transform. In 2016 IEEE International Conference on Real-time Computing and Robotics (RCAR), pp. 411–415. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2013) Lane detection algorithm based on local feature extraction. In 2013 Chinese Automation Congress, pp. 59–64. Cited by: Table 2, §4.
-  (2015-October 20) Video based intelligent vehicle control system. Google Patents. Note: US Patent 9,165,468 Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2001) Efficient implementation of quaternion fourier transform, convolution, and correlation by 2-d complex fft. IEEE Transactions on Signal Processing 49 (11), pp. 2783–2797. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2014) Multispectral iris recognition utilizing hough transform and modified lbp. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1396–1399. Cited by: §1.4.
-  (2016-September 6) Dynamic routing intelligent vehicle enhancement system. Google Patents. Note: US Patent 9,435,652 Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (1997) The discrete quaternion fourier transform. Cited by: §3.
-  (1998) Colour image edge detector based on quaternion convolution. Electronics Letters 34 (10), pp. 969–971. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2015) Real-time illumination invariant lane detection for lane departure warning system. Expert Systems with Applications 42 (4), pp. 1816–1824. Cited by: Table 2, §4, A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2014) Lane-mark extraction for automobiles under complex conditions. Pattern Recognition 47 (8), pp. 2756–2767. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2014) Accurate and robust line segment extraction using minimum entropy with hough transform. IEEE Transactions on Image Processing 24 (3), pp. 813–822. Cited by: §1.4, §1.
-  (2018) Edge detection methods based on modified differential phase congruency of monogenic signal. Multidimensional Systems and Signal Processing 29 (1), pp. 339–359. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.
-  (2013) Gradient-enhancing conversion for illumination-robust lane detection. IEEE Transactions on Intelligent Transportation Systems 14 (3), pp. 1083–1094. Cited by: §4.1, Table 2, §4.
-  (2016) Quaternion collaborative and sparse representation with application to color face recognition. IEEE Transactions on image processing 25 (7), pp. 3287–3302. Cited by: A Robust Lane Detection Associated with Quaternion Hardy Filter.