Auto4D: Learning to Label 4D Objects from Sequential Point Clouds

by   Bin Yang, et al.

In the past few years we have seen great advances in 3D object detection thanks to deep learning methods. However, they typically rely on large amounts of high-quality labels to achieve good performance, which often require time-consuming and expensive work by human annotators. To address this we propose an automatic annotation pipeline that generates accurate object trajectories in 3D (ie, 4D labels) from LiDAR point clouds. Different from previous works that consider single frames at a time, our approach directly operates on sequential point clouds to combine richer object observations. The key idea is to decompose the 4D label into two parts: the 3D size of the object, and its motion path describing the evolution of the object's pose through time. More specifically, given a noisy but easy-to-get object track as initialization, our model first estimates the object size from temporally aggregated observations, and then refines its motion path by considering both frame-wise observations as well as temporal motion cues. We validate the proposed method on a large-scale driving dataset and show that our approach achieves significant improvements over the baselines. We also showcase the benefits of our approach under the annotator-in-the-loop setting.


Multi-Frame to Single-Frame: Knowledge Distillation for 3D Object Detection

A common dilemma in 3D object detection for autonomous driving is that h...

MoNet: Motion-based Point Cloud Prediction Network

Predicting the future can significantly improve the safety of intelligen...

Drosophila-Inspired 3D Moving Object Detection Based on Point Clouds

3D moving object detection is one of the most critical tasks in dynamic ...

Interactive Object Segmentation in 3D Point Clouds

Deep learning depends on large amounts of labeled training data. Manual ...

Tracking from Patterns: Learning Corresponding Patterns in Point Clouds for 3D Object Tracking

A robust 3D object tracker which continuously tracks surrounding objects...

Vegetation Stratum Occupancy Prediction from Airborne LiDAR 3D Point Clouds

We propose a new deep learning-based method for estimating the occupancy...

Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds

Point clouds, as a form of Lagrangian representation, allow for powerful...