Gated2Depth: Real-time Dense Lidar from Gated Images

by   Tobias Gruber, et al.

We present an imaging framework which converts three images from a gated camera into high-resolution depth maps with depth resolution comparable to pulsed lidar measurements. Existing scanning lidar systems achieve low spatial resolution at large ranges due to mechanically-limited angular sampling rates, restricting scene understanding tasks to close-range clusters with dense sampling. In addition, today's lidar detector technologies, short-pulsed laser sources and scanning mechanics result in high cost, power consumption and large form-factors. We depart from point scanning and propose a learned architecture that recovers high-fidelity dense depth from three temporally gated images, acquired with a flash source and a high-resolution CMOS sensor. The proposed architecture exploits semantic context across gated slices, and is trained on a synthetic discriminator loss without the need of dense depth labels. The method is real-time and essentially turns a gated camera into a low-cost dense flash lidar which we validate on a wide range of outdoor driving captures and in simulations.



page 2

page 3

page 5

page 8


Uncertainty depth estimation with gated images for 3D reconstruction

Gated imaging is an emerging sensor technology for self-driving cars tha...

Gated2Gated: Self-Supervised Depth Estimation from Gated Images

Gated cameras hold promise as an alternative to scanning LiDAR sensors w...

The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement

Modern smartphones can continuously stream multi-megapixel RGB images at...

A MEMS-based Foveating LIDAR to enable Real-time Adaptive Depth Sensing

Most active depth sensors sample their visual field using a fixed patter...

Low-cost Retina-like Robotic Lidars Based on Incommensurable Scanning

High performance lidars are essential in autonomous robots such as self-...

Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues

Today's state-of-the-art methods for 3D object detection are based on li...

Learning Super-resolved Depth from Active Gated Imaging

Environment perception for autonomous driving is doomed by the trade-off...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.