3D-VField: Learning to Adversarially Deform Point Clouds for Robust 3D Object Detection

by   Alexander Lehner, et al.

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness on out-of-distribution and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by taking into account deformed point clouds during training. We achieve this with 3D-VField: a novel method that plausibly deforms objects via vectors learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferrable, sample-independent and preserve shape smoothness and occlusions. By augmenting normal samples with the deformations produced by these vector fields during training, we significantly improve robustness against differently shaped objects, such as damaged/deformed cars, even while training only on KITTI. Towards this end, we propose and share open source CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the high generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.



There are no comments yet.


page 8

page 9

page 10

page 11

page 12

page 16


3D Object Detection From LiDAR Data Using Distance Dependent Feature Extraction

This paper presents a new approach to 3D object detection that leverages...

StarNet: Targeted Computation for Object Detection in Point Clouds

LiDAR sensor systems provide high resolution spatial information about t...

DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes

We propose DOPS, a fast single-stage 3D object detection method for LIDA...

Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

Advances in LiDAR sensors provide rich 3D data that supports 3D scene un...

Frustum PointNets for 3D Object Detection from RGB-D Data

While object recognition on 2D images is getting more and more mature, 3...

TANet++: Triple Attention Network with Filtered Pointcloud on 3D Detection

TANet is one of state-of-the-art 3D object detection method on KITTI and...

Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection

In this work, we propose a novel method termed Frustum ConvNet (F-ConvNe...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.