DeepAI AI Chat
Log In Sign Up

End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection

04/07/2020
by   Rui Qian, et al.
cornell university
The Ohio State University
6

Reliable and accurate 3D object detection is a necessity for safe autonomous driving. Although LiDAR sensors can provide accurate 3D point cloud estimates of the environment, they are also prohibitively expensive for many settings. Recently, the introduction of pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. However, so far these two networks have to be trained separately. In this paper, we introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end. The resulting framework is compatible with most state-of-the-art networks for both tasks and in combination with PointRCNN improves over PL consistently across all benchmarks – yielding the highest entry on the KITTI image-based 3D object detection leaderboard at the time of submission. Our code will be made available at https://github.com/mileyan/pseudo-LiDAR_e2e.

READ FULL TEXT

page 1

page 3

page 4

page 7

page 8

page 13

page 14

06/14/2019

Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving

Detecting objects such as cars and pedestrians in 3D plays an indispensa...
09/30/2022

PointPillars Backbone Type Selection For Fast and Accurate LiDAR Object Detection

3D object detection from LiDAR sensor data is an important topic in the ...
03/04/2023

Virtual Sparse Convolution for Multimodal 3D Object Detection

Recently, virtual/pseudo-point-based 3D object detection that seamlessly...
12/10/2019

Learning Depth-Guided Convolutions for Monocular 3D Object Detection

3D object detection from a single image without LiDAR is a challenging t...
11/24/2021

SM3D: Simultaneous Monocular Mapping and 3D Detection

Mapping and 3D detection are two major issues in vision-based robotics, ...
04/13/2021

VR3Dense: Voxel Representation Learning for 3D Object Detection and Monocular Dense Depth Reconstruction

3D object detection and dense depth estimation are one of the most vital...
08/11/2020

Rethinking Pseudo-LiDAR Representation

The recently proposed pseudo-LiDAR based 3D detectors greatly improve th...

Code Repositories

pseudo-LiDAR_e2e

pseudo-LiDAR_e2e


view repo