Learning-Based Framework for Camera Calibration with Distortion Correction and High Precision Feature Detection

I Introduction

Camera calibration is crucial for many robotic applications, such as visual odometry [1], 3D scene reconstruction [2] and autonomous driving [3]. Especially, in some industrial and medical applications [4], precision and robustness are of utmost importance, so camera calibration in such systems has a significant impact on the overall performance.

Fig. 1: Overview of the framework, which includes three main parts. First, radial distortion is removed by correction model inferred from distorted images by our network (green part). The chessboard corner detection part consists of learning-based corner encoding and specially-designed corner decoding (blue part). After high precision sub-pixel corner coordinates are obtained, stable parameter estimation owning to the RANSAC procedure is performed to achieve accurate camera parameters (gray part).

The most widely-used camera calibration toolboxes [5, 6] are built based on Zhang’s technique [7]. Usually chessboard images are captured to calculate camera parameters according to the established feature correspondences between 3D world and 2D images. This pipeline is flexible and easy to implement. Building a precise and robust calibration system, however, is still a challenging problem, mainly due to the following issues:

Inexact detection. Sub-pixel feature localization is hard to achieve, especially in scenario with noise and bad illumination.
Radial distortion. Severe radial lens distortion may result in calibration failure.
Sub-optimal estimation. Purely algebraic optimization of re-projection error leads to sub-optimal and unstable calibration results.

For the first issue, many early works [8, 9, 10]

make efforts to sub-pixel feature localization using hand-crafted features, but are not robust when confronted with noises. As the rapid advance of deep learning, Convolution Neural Network (CNN) is introduced

[11, 12, 13] to detect chessboard corners but still limited in pixel level precision. To address this issue, we extend deep learning paradigm to achieve sub-pixel feature localization by adopting an efficient encoding-decoding scheme which has been proved efficient in body joint estimation [14]. However, unlike joints, the chessboard corners preserve two peculiarities: (1) commonly-seen pattern resulting in fake corner detection in background and (2) collineation under projective transformation without considering distortion. Therefore, we propose a specifically-designed sub-pixel coordinate decoding algorithm, in which Gaussian surface fitting is applied to get initial sub-pixel coordinates and then distribution-aware outlier rejection is performed to eliminate unreliable corners. Moreover, the collineation refinement is adopted to get the final corner coordinates after inlier candidates are selected.

The second issue is about lens distortion correction, which is a non-trivial problem in camera calibration, as real cameras often exhibit lens distortion, especially radial distortion [15]. Classical methods [16, 17, 7]

estimate camera and distortion parameters simultaneously by iterative optimization. However, ambiguity is introduced in this way as the parameters are tangled together leading to failure under severe distortion. Fish-eye image rectification provides a positive example to handle this issue, where the same distortion is handled by deep neural network successfully

[18, 19, 20], as the distortion is related to the curvature of the straight scene lines. Inspired by this, we adopt CNN to infer the correction parameters from distorted images but adopt a more practical distortion model different from previous works. As the corner detection accuracy suffers from radial distortion and our collineation refinement assumes no distortion, so the distortion correction is taken as the first step in the proposed framework.

The third issue is caused by purely algebraic optimization which may lead to unreasonable calibration results. Some works [21, 22] propose geometry-based algorithms to address this issue, but not complete enough to be widely used. Random Sample Consensus (RANSAC) [23] algorithm has been introduced into calibration to eliminate outliers to enhance the stability of parameter estimation [24, 25]. We also adopt a RANSAC-based calibration procedure to improve the robustness of parameter estimation. Different from previous work, as outlier rejection already gets involved in our model, the proposed RANSAC-based algorithm aims at searching for the optimal camera model.

In sum, the critical components of a calibration system, distortion correction, feature detection and parameter estimation, are reforged and integrated as a novel and efficient calibration framework (Fig. 1). The contributions can be summarized as follows.

A novel camera calibration framework is proposed, which includes radial distortion correction, sub-pixel feature detection and robust parameter estimation.
We design a sub-pixel coordinate decoding algorithm with outlier rejection and collineation refinement to cooperate with the learning coordinate encoding method.
The new framework achieves precise results on both synthetic and real data, outperforming widely-used methods by a noteworthy margin.

Ii Related work

We review literature concerning the three parts of calibration divided by our framework.

Corner Detection. Firstly, corner detectors such as Harris[26] are adopted, but they are sensitive to noise. Lucchese et al.[8] propose a sub-pixel corner detection algorithm utilizing saddle points based on the chessboard corner feature and it is extended by Placht et al.[9] through a surface fitting refinement. The widely-used findChessboadrCorners in OpenCV[5] refines coordinates to sub-pixel according to gray distribution constraints. In [10], corner coordinates are refined to sub-pixel accuracy based on the chessboard structure. While these methods achieve sub-pixel precision, heavily relying on hand-crafted features leads to the lack of robustness. Recently, some detection algorithms based on learning features are proposed [11, 12, 13]. They are robust against noise but trapped in pixel level accuracy. On the other hand, learning coordinate encoding with refinement decoding methods [14, 27] boost the performance of body joint localization which are suggestive for this task.

Radial Distortion Correction. Classical algorithms integrate distortion in camera model and solve it by non-linear optimization techniques [16]. Tsai [17] estimates distortion parameters by an iterative scheme. Zhang’s technique [7] corrects the distortion by minimizing the reprojection error. Although this method works well under slight radial distortion, they may end up with a bad solution when the distortion is severe. In similar applications like fisheye image rectification, some researchers [28, 18, 19] adopt deep CNN to learn distortion parameters and achieve decent performance.

Parameter Estimation. Widely-used Zhang’s technique [7] first solves the initial guess of parameters based on the correspondence between the real world and image. Then these parameters are refined by minimizing the reprojection error. However, purely algebraic optimization is unstable leading to suboptimal calibration results. To address this issue, geometric-based methods [22, 21] are proposed. Moreover, RANSAC-based algorithms [25, 24] are proposed to achieve robust parameter estimation by excluding unreliable images or corners.

Fig. 2: Examples of corner encoding outliers. The left image (real) and its heatmap are related to lost corners and the right ones (synthetic) are fake corners.

Iii The proposed calibration Framework

In this section, we introduce our camera calibration framework by describing each component.

Iii-a Radial Distortion Correction

As simultaneous parameter estimation and distortion correction not only increase calibration effort but also reduce calibration precision, distortion correction is first performed separately in our framework. Inspired by previous works [18, 20] performing distortion correction on general images taken by fisheye camera, we apply learning approach to correct camera distortion. Unlike general images, chessboard images contain much more straight lines which are wrapped due to distortion, then the correlation between curved line features and distortion is supposed to be intensive. Therefore, we adopt a simple CNN-based encoder with regression layers to infer correction parameters from distorted images. Unlike previous work, we apply a more practical distortion model, the radial model ( $r_{d} = r_{c} (1 + k_{1} r_{c}^{2})$ ), which is symmetric and flexible [15]. Its symmetric property[15] maintains consistency between the distortion and correction, which is another radial model with higher order.

r_{c} = r_{d} (k_{0}^{^{'}} + k_{1}^{^{'}} r_{d} + k_{2}^{^{'}} r_{d}^{2} + \dots)

(1)

Its flexibility allows us to choose appropriate correction parameter numbers according to the specific distortion. To train the network, massive distorted chessboard images are generated and robust sampling grid loss [20] is adopted.

Iii-B Chessboard Corner Detection

Learning heatmap encoding and decoding methods provide decent performance in body joint localization[14, 27]. Inspired by these methods, we proposed a similar but more specific chessboard corner detection technique. Firstly, a CNN-based encoder-decoder network is adopted to transform the corrected chessboard image into a coordinate heatmap. Besides, heatmap modulation [14]

is utilized to improve the encoding performance. This process can encode sub-pixel corner coordinates as a 2-dimensional Gaussian distribution (

G

) is centered at the labelled coordinate of each corner in the ground truth heatmap used to train the network. However, unlike body joints, the chessboard corners have unique properties: (i) The black and white feature is commonly-seen resulting in fake corners in the background. (ii) As camera projective transformation (without distortion) is a collineation [29], the chessboard corners in the image should be the intersections of groups of lines. Therefore, we propose a specially-designed coordinate decoding algorithm. First, Gaussian surface fitting algorithm is applied on each corner’s distribution of heatmaps to get the center

μ

and the variance

σ

arg min μ, σ {∥ G (μ, σ) - C ∥}_{2}^{2}

(2)

where $C$ is the point set distribution of a single corner in the transformed heatmap. However, some abnormal distributions in heatmaps occur in practice. They are caused by similar patterns in background (fake corners) and encoding failure due to occlusion, extreme pose and bad lighting (lost corners) which can be seen in Fig. 2. In order to exclude these fake and lost corners, we propose the distribution-aware outlier rejection. This process determines outliers by comparing each distribution variance $σ$ with the ground truth one in training data. After inliers are selected, collineation refinement is proposed not only to refine the sub-pixel coordinates but also to recover the lost corners. Due to the distortion being removed first by our framework, this refinement utilizes initial corner candidates to fit lines and then calculate the final coordinates by intersecting these lines.

Iii-C Parameter Estimation

In order to achieve robust parameter estimation, we propose a simple yet effective parameter estimation method which improves Zhang’s technique [7] by RANSAC. Different with early works [25, 24], the proposed parameter estimation is aimed at searching for the best camera model which keeps consistency in reprojection error, because our corner detection part already includes outlier rejection. This method can be illuminated as follows:

Choose some of the images randomly to estimate parameters based on Zhang’s technique.
Calculate the reprojection error of all images and determine inliers whose reprojection errors are less than the threshold.
Output the parameters if the inliers number is big enough; otherwise repeat the above steps.

Iv Experimental Results

In this section, the performance of the proposed camera calibration framework is evaluated on both synthetic and real data. We also construct experiments to demonstrate the capabilities of our distortion correction part and corner detection part respectively.

Iv-a Dataset and Metric

To train our network, we generate massive chessboard images with ground truth corner heatmaps and camera parameters. Moreover, noise, bad lighting, distortion and fake background using TUM dataset [30] are applied on images as data augmentation. Specifically, the image distortion level is decided by parameter $k_{1}$ [20].

As camera calibration is aimed at intrinsic parameter estimation, we use the metrics related to focal length (FL) and principle points (PP) which can be defined as:

E_{F L} = ∥ F L_{G T} - F L_{e s t} ∥

(3)

E_{P P} = ∥ P P_{G T} - P P_{e s t} ∥

(4)

E_{I P} = \frac{1}{2} (E_{F L} + E_{P P})

(5)

where $E_{I P}$ means intrinsic parameter error.

Iv-B Calibration Performance

To extensively evaluate our framework, we conduct three kinds of calibration experiments: calibration under noise and bad lighting, calibration under distortion and calibration on real data. Our framework is compared with two widely-used calibration toolboxes: OpenCV [5] and Matlab [6].

Calibration under noise and bad lighting. To demonstrate the robustness and accuracy of our framework in terms of environmental noise like low sensor resolution and uneven illumination, we perform calibration on synthetic data with or without extra Gaussian noise and uneven brightness. The average results of 50 independent trials are summarized in Table II. It can be seen that our framework not only achieves the best accuracy but also maintains the best robustness. For instance, the intrinsic parameter error of our system reduces $68 %$ than OpenCV and $51 %$

than Matlab on dataset without noise and bad lighting. What’s more, the performances of our framework under noise and bad lighting are much better than the other methods, as the learning approaches are robust against environmental noise. On the other hand, the lower standard deviations manifest the stability of our framework owing to the RANSAC procedure.

Calibration under distortion. In this part, we conduct calibration experiments on distorted images to evaluate our framework performance. To extensively test our framework under different distortion levels, we change the parameter $k_{1}$ from $- 0.2$ to $- 0.5$ . Fig.3 shows the examples of distorted images along with the images corrected by our correction part. For each distortion level, 50 independent trials of calibration are performed and average results are shown in Fig. 4

. It can be seen that the calibration errors of the two widely-used methods are increasing with the distortion level. However, our system keeps precise results under all distortion levels, which is probably due to our efficient distortion correction part.

Fig. 3: Examples of images under different degrees of distortion (top) along with the corrected images (bottom) acquired by our method. The image is more distorted when the parameter $k_{1}$ is farther from 0. The correction results can be evaluated by the corrected lines.

	Ours	Matlab	OpenCV
$f_{x}$	781.06	783.24	776.43
$f_{y}$	1043.11	1047.0	1038.32
$p_{x}$	744.51	756.33	803.22
$p_{y}$	539.17	561.42	506.79
RPE	0.38	0.37	0.87
STD	0.21	0.87	1.75

TABLE I: Comparison of calibration using real data.

[b]

TABLE II: Comparison of camera calibration with noise and bad lighting.

Noise	Lighting	Calibration errors(pixels)
		Ours				Matlab				OpenCV
		$E_{F L}$	$E_{P P}$	$E_{I P}$	$σ$ ^*	$E_{F L}$	$E_{P P}$	$E_{I P}$	$σ$ ^*	$E_{F L}$	$E_{P P}$	$E_{I P}$	$σ$ ^*
$\times$	normal	0.58	1.03	0.81	0.08	2.83	0.74	1.78	0.87	3.88	1.16	2.52	1.05
✓	normal	0.42	0.94	0.68	0.22	2.41	0.63	1.52	3.37	9.40	4.78	7.09	5.76
$\times$	bad	0.84	0.64	1.04	0.27	2.38	3.48	1.27	1.79	5.65	9.73	1.56	7.78
✓	bad	0.94	1.43	1.18	0.35	11.24	7.13	9.18	5.85	13.43	10.33.	11.88	8.73

The standard deviation of $E_{I P}$ .

Fig. 4: Comparison of calibration w.r.t distortion.

Calibration on real data. To evaluate the performance of our system under realistic conditions, we perform calibration on a HIKROBOT MV-CA016-10GM camera with resolution of $1440 \times 1080$ by a $12 \times 8$ chessboard. We repeat 20 times of calibration with different combinations of chessboard poses and get the average results of intrinsic parameters, reprojection error and the standard deviation of parameters (average of the 4 parameters). The results compared with two widely-used calibration toolboxes are provided in Table. I. We can observe that the three systems produce similar results and our results are closer to Matlab instead of the OpenCV. While the reprojection error of our system is almost the same as Matlab, our framework is more accurate because the principle point coordinates should be close to the center of the image according to the prior knowledge about the real camera. Moreover, the lower standard deviation also proves the higher stability of our framework.

Iv-C Corner Detection Accuracy

As calibration benefits from precise chessboard corner coordinates, the accuracy of our corner detection method is tested on synthetic data in this subsection. Compared with both feature-based [5, 10] and learning-based [27] corner detection methods with sub-pixel accuracy, we conduct experiments on synthetic images with different configurations including extra noise, bad lighting and radial distortion ( $k_{1} = - 0.2$ ). Each configuration contains 2K chessboard images with ground truth sub-pixel corner coordinates. The results are shown in Table. III. It can be seen that our method gets the highest precision on different images, which is consistent with the calibration experiments and proves the precision of our method.

Corner Detection Accuracy (pixels)
Methods	Image Configuration
Methods	Noise	Bad Lighting	Distortion
OpenCV [5]	2.23	2.43	2.79
libcb [10]	1.91	2.72	2.76
Heatmap [27]	1.35	1.46	1.77
Ours	0.78	0.51	0.71

TABLE III: Corner detection accuracy under different images

Iv-D Distortion Correction performance

To measure the contribution of our distortion correction part, the calibration accuracy of our framework with and without this part are compared. What’s more, we combine our distortion correction part with Matlab and compare its accuracy with the original one. The experiments are conducted on our synthetic dataset whose distortion parameter $k_{1} = - 0.3$ and the results are provided in Table IV

. It can be seen that the distortion correction network improves the accuracy a lot in our framework. Matlab combined with the distortion correction part gets accuracy improvement which is yet limited because the distortion correction brings into noise caused by image interpolation. As our corner detection part can achieve accurate results against this noise, our framework can handle the distortion well.

[t] Ours Matlab Errors $E_{F L}$ $E_{P P}$ $E_{I P}$ $E_{F L}$ $E_{P P}$ $E_{I P}$ no DC¹ 1.06 3.9 2.43 4.99 6.88 5.93 DC² 0.42 1.16 0.79 4.42 4.39 4.41

TABLE IV: Calibration with/without Distortion correction

Calibration without our distortion correction part.
Calibration with our distortion correction part.

V Conclusions

In this paper, we propose a novel camera calibration framework containing three parts: distortion correction, corner detection and parameter estimation. This framework integrates learning-based approaches with traditional methods to achieve efficient distortion correction, accurate sub-pixel feature detection and stable parameter estimation. This framework surpasses other widely-used calibration methods by a large margin in terms of accuracy on both synthetic and real dataset. Additionally, extensive experiments prove the robustness of this framework against noise, lighting and radial distortion. Besides, the corner detection and distortion correction part are evaluated respectively where decent results manifest the contribution of these parts to our framework.

References

[1] J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 3, pp. 611–625, 2017.
[2] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309–1332, 2016.
[3] P. F. Martins, H. Costelha, L. C. Bento, and C. Neves, “Monocular camera calibration for autonomous driving — a comparative study,” in 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2020, pp. 306–311.
[4] J. Barreto, J. Roquette, P. Sturm, and F. Fonseca, “Automatic camera calibration applied to medical endoscopy,” in BMVC 2009-20th British Machine Vision Conference. The British Machine Vision Association (BMVA), 2009, pp. 1–10.
[5] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.
[6] MATLAB, version 7.10.0 (R2010a). Natick, Massachusetts: The MathWorks Inc., 2010.
[7] Z. Zhang, “A flexible new technique for camera calibration,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 22, pp. 1330 – 1334, 12 2000.
[8] L. Lucchese and S. Mitra, “Using saddle points for subpixel feature detection in camera calibration targets,” in Asia-Pacific Conference on Circuits and Systems, vol. 2, 2002, pp. 191–195 vol.2.
[9] S. Placht, P. Fürsattel, E. A. Mengue, H. Hofmann, C. Schaller, M. Balda, and E. Angelopoulou, “Rochade: Robust checkerboard advanced detection for camera calibration,” in Computer Vision – ECCV 2014, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, 2014, pp. 766–779.
[10] A. Geiger, F. Moosmann, O. Car, and B. Schuster, “Automatic camera and range sensor calibration using a single shot,” in International Conference on Robotics and Automation (ICRA), St. Paul, USA, May 2012.
[11]
S. Donné, J. De Vylder, B. Goossens, and W. Philips, “MATE: Machine Learning for Adaptive Calibration Template Detection,”
Sensors, vol. 16, no. 11, pp. 1858–17, Nov. 2016.
[12] B. Chen, C. Xiong, and Q. Zhang, “CCDN: Checkerboard Corner Detection Network for Robust Camera Calibration,” in Intelligent Robotics and Applications. Cham: Springer, Cham, Aug. 2018, pp. 324–334.
[13] H. Wu and Y. Wan, “A highly accurate and robust deep checkerboard corner detector,” Electronics Letters, vol. 57, no. 8, pp. 317–320, 2021.
[14]
F. Zhang, X. Zhu, H. Dai, M. Ye, and C. Zhu, “Distribution-aware coordinate representation for human pose estimation,” in
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
, 2020, pp. 7093–7102.
[15] Z. Tang, R. Grompone von Gioi, P. Monasse, and J.-M. Morel, “A Precision Analysis of Camera Distortion Models,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2694–2704, Sept. 2020.
[16] J. Salvi et al., An approach to coded structured light to obtain three dimensional information. Universitat de Girona, 1998.
[17] R. Tsai, “A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses,” IEEE Journal on Robotics and Automation, vol. 3, no. 4, pp. 323–344, 1987.
[18] Z. Xue, N. Xue, G.-S. Xia, and W. Shen, “Learning to calibrate straight lines for fisheye image rectification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[19] X. Yin, X. Wang, J. Yu, M. Zhang, P. Fua, and D. Tao, “Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 469–484.
[20] H. Zhao, Y. Shi, X. Tong, and X. Y. International, “A Simple Yet Effective Pipeline For Radial Distortion Correction,” ieeexplore.ieee.org, pp. 878–882, 2020.
[21] R. Sagawa and Y. Yagi, “Accurate calibration of intrinsic camera parameters by observing parallel light pairs,” 06 2008, pp. 1390 – 1397.
[22] J.-H. Chuang, C.-H. Ho, A. Umam, H.-Y. Chen, J.-N. Hwang, and T.-A. Chen, “Geometry-based camera calibration using closed-form solution of principal line,” IEEE Transactions on Image Processing, vol. 30, pp. 2599–2610, 2021.
[23] M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, vol. 24, no. 6, p. 381–395, June 1981. [Online]. Available: https://doi.org/10.1145/358669.358692
[24] F. Zhou, Y. Cui, Y. Wang, L. Liu, and H. Gao, “Accurate and robust estimation of camera parameters using ransac,” Optics and Lasers in Engineering, vol. 51, no. 3, pp. 197–212, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0143816612003016
[25] Y. Lv, J. Feng, Z. Li, W. Liu, and J. Cao, “A new robust 2d camera calibration method using ransac,” Optik, vol. 126, no. 24, pp. 4910–4915, 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0030402615011869
[26] C. Harris and M. Stephens, “A combined corner and edge detector,” in Alvey Vision Conference, 1988.
[27] J. M. Graving, D. Chae, H. Naik, L. Li, B. Koger, B. R. Costelloe, and I. D. Couzin, “Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning,” Elife, vol. 8, p. e47994, 2019.
[28] J. Rong, S. Huang, Z. Shang, and X. Ying, “Radial lens distortion correction using convolutional neural networks trained with synthesized images,” in ACCV, 03 2017, pp. 35–49.
[29] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press, 2004.
[30] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.

References

[1] J. Engel, V. Koltun, and D. Cremers, “Direct sparse odometry,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 3, pp. 611–625, 2017.
[2] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, present, and future of simultaneous localization and mapping: Toward the robust-perception age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309–1332, 2016.
[3] P. F. Martins, H. Costelha, L. C. Bento, and C. Neves, “Monocular camera calibration for autonomous driving — a comparative study,” in 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2020, pp. 306–311.
[4] J. Barreto, J. Roquette, P. Sturm, and F. Fonseca, “Automatic camera calibration applied to medical endoscopy,” in BMVC 2009-20th British Machine Vision Conference. The British Machine Vision Association (BMVA), 2009, pp. 1–10.
[5] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools, 2000.
[6] MATLAB, version 7.10.0 (R2010a). Natick, Massachusetts: The MathWorks Inc., 2010.
[7] Z. Zhang, “A flexible new technique for camera calibration,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 22, pp. 1330 – 1334, 12 2000.
[8] L. Lucchese and S. Mitra, “Using saddle points for subpixel feature detection in camera calibration targets,” in Asia-Pacific Conference on Circuits and Systems, vol. 2, 2002, pp. 191–195 vol.2.
[9] S. Placht, P. Fürsattel, E. A. Mengue, H. Hofmann, C. Schaller, M. Balda, and E. Angelopoulou, “Rochade: Robust checkerboard advanced detection for camera calibration,” in Computer Vision – ECCV 2014, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. Cham: Springer International Publishing, 2014, pp. 766–779.
[10] A. Geiger, F. Moosmann, O. Car, and B. Schuster, “Automatic camera and range sensor calibration using a single shot,” in International Conference on Robotics and Automation (ICRA), St. Paul, USA, May 2012.
[11] S. Donné, J. De Vylder, B. Goossens, and W. Philips, “MATE: Machine Learning for Adaptive Calibration Template Detection,” Sensors, vol. 16, no. 11, pp. 1858–17, Nov. 2016.
[12] B. Chen, C. Xiong, and Q. Zhang, “CCDN: Checkerboard Corner Detection Network for Robust Camera Calibration,” in Intelligent Robotics and Applications. Cham: Springer, Cham, Aug. 2018, pp. 324–334.
[13] H. Wu and Y. Wan, “A highly accurate and robust deep checkerboard corner detector,” Electronics Letters, vol. 57, no. 8, pp. 317–320, 2021.
[14] F. Zhang, X. Zhu, H. Dai, M. Ye, and C. Zhu, “Distribution-aware coordinate representation for human pose estimation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 7093–7102.
[15] Z. Tang, R. Grompone von Gioi, P. Monasse, and J.-M. Morel, “A Precision Analysis of Camera Distortion Models,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2694–2704, Sept. 2020.
[16] J. Salvi et al., An approach to coded structured light to obtain three dimensional information. Universitat de Girona, 1998.
[17] R. Tsai, “A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses,” IEEE Journal on Robotics and Automation, vol. 3, no. 4, pp. 323–344, 1987.
[18] Z. Xue, N. Xue, G.-S. Xia, and W. Shen, “Learning to calibrate straight lines for fisheye image rectification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[19] X. Yin, X. Wang, J. Yu, M. Zhang, P. Fua, and D. Tao, “Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 469–484.
[20] H. Zhao, Y. Shi, X. Tong, and X. Y. International, “A Simple Yet Effective Pipeline For Radial Distortion Correction,” ieeexplore.ieee.org, pp. 878–882, 2020.
[21] R. Sagawa and Y. Yagi, “Accurate calibration of intrinsic camera parameters by observing parallel light pairs,” 06 2008, pp. 1390 – 1397.
[22] J.-H. Chuang, C.-H. Ho, A. Umam, H.-Y. Chen, J.-N. Hwang, and T.-A. Chen, “Geometry-based camera calibration using closed-form solution of principal line,” IEEE Transactions on Image Processing, vol. 30, pp. 2599–2610, 2021.
[23] M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, vol. 24, no. 6, p. 381–395, June 1981. [Online]. Available: https://doi.org/10.1145/358669.358692
[24] F. Zhou, Y. Cui, Y. Wang, L. Liu, and H. Gao, “Accurate and robust estimation of camera parameters using ransac,” Optics and Lasers in Engineering, vol. 51, no. 3, pp. 197–212, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0143816612003016
[25] Y. Lv, J. Feng, Z. Li, W. Liu, and J. Cao, “A new robust 2d camera calibration method using ransac,” Optik, vol. 126, no. 24, pp. 4910–4915, 2015. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0030402615011869
[26] C. Harris and M. Stephens, “A combined corner and edge detector,” in Alvey Vision Conference, 1988.
[27] J. M. Graving, D. Chae, H. Naik, L. Li, B. Koger, B. R. Costelloe, and I. D. Couzin, “Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning,” Elife, vol. 8, p. e47994, 2019.
[28] J. Rong, S. Huang, Z. Shang, and X. Ying, “Radial lens distortion correction using convolutional neural networks trained with synthesized images,” in ACCV, 03 2017, pp. 35–49.
[29] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press, 2004.
[30] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.

Learning-Based Framework for Camera Calibration with Distortion Correction and High Precision Feature Detection

High-precision camera distortion measurements with a "calibration harp"

Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation

Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve

Visualizing and Alleviating the Effect of Radial Distortion on Camera Calibration Using Principal Lines

Inferring bias and uncertainty in camera calibration

Self-Calibrating Neural Radiance Fields

A calibration framework for high-resolution hydrological models using a multiresolution and heterogeneous strategy

Code Repositories

CCS

I Introduction

Ii Related work

Iii The proposed calibration Framework

Iii-a Radial Distortion Correction

Iii-B Chessboard Corner Detection

Iii-C Parameter Estimation

Iv Experimental Results

Iv-a Dataset and Metric

Iv-B Calibration Performance

Iv-C Corner Detection Accuracy

Iv-D Distortion Correction performance

V Conclusions

References

References

Learning-Based Framework for Camera Calibration with Distortion Correction and High Precision Feature Detection

Related Research

High-precision camera distortion measurements with a "calibration harp"

Learning Markerless Robot-Depth Camera Calibration and End-Effector Pose Estimation

Why Having 10,000 Parameters in Your Camera Model is Better Than Twelve

Visualizing and Alleviating the Effect of Radial Distortion on Camera Calibration Using Principal Lines

Inferring bias and uncertainty in camera calibration

Self-Calibrating Neural Radiance Fields

A calibration framework for high-resolution hydrological models using a multiresolution and heterogeneous strategy

Code Repositories

CCS

I Introduction

Ii Related work

Iii The proposed calibration Framework

Iii-a Radial Distortion Correction

Iii-B Chessboard Corner Detection

Iii-C Parameter Estimation

Iv Experimental Results

Iv-a Dataset and Metric

Iv-B Calibration Performance

Iv-C Corner Detection Accuracy

Iv-D Distortion Correction performance

V Conclusions

References

References