DeepAI AI Chat
Log In Sign Up

Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

by   Yan Wang, et al.
cornell university

3D object detection is an essential task in autonomous driving. Recent techniques excel with highly accurate detection rates, provided the 3D input data is obtained from precise but expensive LiDAR technology. Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies --- a gap that is commonly attributed to poor image-based depth estimation. However, in this paper we argue that data representation (rather than its quality) accounts for the majority of the difference. Taking the inner workings of convolutional neural networks into consideration, we propose to convert image-based depth maps to pseudo-LiDAR representations --- essentially mimicking LiDAR signal. With this representation we can apply different existing LiDAR-based detection algorithms. On the popular KITTI benchmark, our approach achieves impressive improvements over the existing state-of-the-art in image-based performance --- raising the detection accuracy of objects within 30m range from the previous state-of-the-art of 22% to an unprecedented 74%. At the time of submission our algorithm holds the highest entry on the KITTI 3D object detection leaderboard for stereo image based approaches.


page 1

page 3

page 4

page 8

page 10

page 11

page 12


Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving

Detecting objects such as cars and pedestrians in 3D plays an indispensa...

Accurate and Real-time Pseudo Lidar Detection: Is Stereo Neural Network Really Necessary?

The proposal of Pseudo-Lidar representation has significantly narrowed t...

Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking

Monocular image-based 3D perception has become an active research area i...

Orthographic Feature Transform for Monocular 3D Object Detection

3D object detection from monocular images has proven to be an enormously...

Neighbor-Vote: Improving Monocular 3D Object Detection through Neighbor Distance Voting

As cameras are increasingly deployed in new application domains such as ...

Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

Pseudo-LiDAR 3D detectors have made remarkable progress in monocular 3D ...

Rethinking Pseudo-LiDAR Representation

The recently proposed pseudo-LiDAR based 3D detectors greatly improve th...

Code Repositories


(CVPR 2019) Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

view repo


Pseudo-LiDAR End to End is a 3D object detector which can directly generate accurate 3D bounding box from binocular camera images in a end-to-end method. (Not the version in Kitti leaderboard)

view repo


LiDAR Guided Boundary Bleeding Refinement for Depth Map

view repo


Review my latest deep learning research papers

view repo