RegNet: Multimodal Sensor Registration Using Deep Neural Networks

07/11/2017
by   Nick Schneider, et al.
0

In this paper, we present RegNet, the first deep convolutional neural network (CNN) to infer a 6 degrees of freedom (DOF) extrinsic calibration between multimodal sensors, exemplified using a scanning LiDAR and a monocular camera. Compared to existing approaches, RegNet casts all three conventional calibration steps (feature extraction, feature matching and global regression) into a single real-time capable CNN. Our method does not require any human interaction and bridges the gap between classical offline and target-less online calibration approaches as it provides both a stable initial estimation as well as a continuous online correction of the extrinsic parameters. During training we randomly decalibrate our system in order to train RegNet to infer the correspondence between projected depth measurements and RGB image and finally regress the extrinsic calibration. Additionally, with an iterative execution of multiple CNNs, that are trained on different magnitudes of decalibration, our approach compares favorably to state-of-the-art methods in terms of a mean calibration error of 0.28 degrees for the rotational and 6 cm for the translation components even for large decalibrations up to 1.5 m and 20 degrees.

READ FULL TEXT

page 1

page 4

page 6

page 7

research
12/27/2020

Lidar and Camera Self-Calibration using CostVolume Network

In this paper, we propose a novel online self-calibration approach for L...
research
03/27/2021

CalibDNN: Multimodal Sensor Calibration for Perception Using Deep Neural Networks

Current perception systems often carry multimodal imagers and sensors su...
research
04/24/2021

CFNet: LiDAR-Camera Registration Using Calibration Flow Network

As an essential procedure of data fusion, LiDAR-camera calibration is cr...
research
03/17/2022

DXQ-Net: Differentiable LiDAR-Camera Extrinsic Calibration Using Quality-aware Flow

Accurate LiDAR-camera extrinsic calibration is a precondition for many m...
research
04/25/2021

Motion based Extrinsic Calibration of a 3D Lidar and an IMU

This work presents a novel extrinsic calibration estimation algorithm be...
research
07/24/2022

Keypoint-less Camera Calibration for Sports Field Registration in Soccer

Sports field registration in broadcast videos is typically interpreted a...
research
12/31/2021

Separation of scales and a thermodynamic description of feature learning in some CNNs

Deep neural networks (DNNs) are powerful tools for compressing and disti...

Please sign up or login with your details

Forgot password? Click here to reset