Frustum Fusion: Pseudo-LiDAR and LiDAR Fusion for 3D Detection

11/08/2021
by   Farzin Negahbani, et al.
0

Most autonomous vehicles are equipped with LiDAR sensors and stereo cameras. The former is very accurate but generates sparse data, whereas the latter is dense, has rich texture and color information but difficult to extract robust 3D representations from. In this paper, we propose a novel data fusion algorithm to combine accurate point clouds with dense but less accurate point clouds obtained from stereo pairs. We develop a framework to integrate this algorithm into various 3D object detection methods. Our framework starts with 2D detections from both of the RGB images, calculates frustums and their intersection, creates Pseudo-LiDAR data from the stereo images, and fills in the parts of the intersection region where the LiDAR data is lacking with the dense Pseudo-LiDAR points. We train multiple 3D object detection methods and show that our fusion strategy consistently improves the performance of detectors.

READ FULL TEXT

page 1

page 2

page 3

page 6

research
06/09/2020

Stereo RGB and Deeper LIDAR Based Network for 3D Object Detection

3D object detection has become an emerging task in autonomous driving sc...
research
03/29/2021

Detecting and Mapping Trees in Unstructured Environments with a Stereo Camera and Pseudo-Lidar

We present a method for detecting and mapping trees in noisy stereo came...
research
03/18/2022

Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion

Current LiDAR-only 3D detection methods inevitably suffer from the spars...
research
11/29/2021

VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion

It has been well recognized that fusing the complementary information fr...
research
04/09/2023

Sparse Dense Fusion for 3D Object Detection

With the prevalence of multimodal learning, camera-LiDAR fusion has gain...
research
11/25/2022

Far3Det: Towards Far-Field 3D Detection

We focus on the task of far-field 3D detection (Far3Det) of objects beyo...
research
03/28/2022

LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection

In this paper, we propose the LiDAR Distillation to bridge the domain ga...

Please sign up or login with your details

Forgot password? Click here to reset