BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

05/27/2022
by   Tingting Liang, et al.
0

Fusing the camera and LiDAR information has become a de-facto standard for 3D object detection tasks. Current methods rely on point clouds from the LiDAR sensor as queries to leverage the feature from the image space. However, people discover that this underlying assumption makes the current fusion framework infeasible to produce any prediction when there is a LiDAR malfunction, regardless of minor or major. This fundamentally limits the deployment capability to realistic autonomous driving scenarios. In contrast, we propose a surprisingly simple yet novel fusion framework, dubbed BEVFusion, whose camera stream does not depend on the input of LiDAR data, thus addressing the downside of previous methods. We empirically show that our framework surpasses the state-of-the-art methods under the normal training settings. Under the robustness training settings that simulate various LiDAR malfunctions, our framework significantly surpasses the state-of-the-art methods by 15.7 28.9 LiDAR malfunction and can be deployed to realistic scenarios without any post-processing procedure. The code is available at https://github.com/ADLab-AutoDrive/BEVFusion.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/30/2022

Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection

There are two critical sensors for 3D perception in autonomous driving, ...
research
07/21/2023

HVDetFusion: A Simple and Robust Camera-Radar Fusion Framework

In the field of autonomous driving, 3D object detection is a very import...
research
06/12/2023

MaskedFusion360: Reconstruct LiDAR Data by Querying Camera Features

In self-driving applications, LiDAR data provides accurate information a...
research
08/23/2019

LiDARTag: A Real-Time Fiducial Tag using Point Clouds

Image-based fiducial markers are widely used in robotics and computer vi...
research
05/23/2022

Graph-theoretical approach to robust 3D normal extraction of LiDAR data

Low dimensional primitive feature extraction from LiDAR point clouds (su...
research
12/12/2022

PathFusion: Path-consistent Lidar-Camera Deep Feature Fusion

Fusing camera with LiDAR is a promising technique to improve the accurac...
research
03/23/2023

Collaboration Helps Camera Overtake LiDAR in 3D Detection

Camera-only 3D detection provides an economical solution with a simple c...

Please sign up or login with your details

Forgot password? Click here to reset