DeepAI AI Chat
Log In Sign Up

MVX-Net: Multimodal VoxelNet for 3D Object Detection

by   Vishwanath A. Sindagi, et al.
Johns Hopkins University
Apple Inc.

Many recent works on 3D object detection have focused on designing neural network architectures that can consume point cloud data. While these approaches demonstrate encouraging performance, they are typically based on a single modality and are unable to leverage information from other modalities, such as a camera. Although a few approaches fuse data from different modalities, these methods either use a complicated pipeline to process the modalities sequentially, or perform late-fusion and are unable to learn interaction between different modalities at early stages. In this work, we present PointFusion and VoxelFusion: two simple yet effective early-fusion approaches to combine the RGB and point cloud modalities, by leveraging the recently introduced VoxelNet architecture. Evaluation on the KITTI dataset demonstrates significant improvements in performance over approaches which only use point cloud data. Furthermore, the proposed method provides results competitive with the state-of-the-art multimodal algorithms, achieving top-2 ranking in five of the six bird's eye view and 3D detection categories on the KITTI benchmark, by using a simple single stage network.


page 1

page 2

page 4

page 6


Temp-Frustum Net: 3D Object Detection with Temporal Fusion

3D object detection is a core component of automated driving systems. St...

CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection

There have been significant advances in neural networks for both 3D obje...

Adaptive and Azimuth-Aware Fusion Network of Multimodal Local Features for 3D Object Detection

This paper focuses on the construction of stronger local features and th...

Part-A^2 Net: 3D Part-Aware and Aggregation Neural Network for Object Detection from Point Cloud

In this paper, we propose the part-aware and aggregation neural network ...

Multimodal Object Detection via Bayesian Fusion

Object detection with multimodal inputs can improve many safety-critical...

PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation

We present PointFusion, a generic 3D object detection method that levera...

A Sequence Agnostic Multimodal Preprocessing for Clogged Blood Vessel Detection in Alzheimer's Diagnosis

Successful identification of blood vessel blockage is a crucial step for...