Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

by   Norman Müller, et al.

Multi-object tracking from RGB-D video sequences is a challenging problem due to the combination of changing viewpoints, motion, and occlusions over time. We observe that having the complete geometry of objects aids in their tracking, and thus propose to jointly infer the complete geometry of objects as well as track them, for rigidly moving objects over time. Our key insight is that inferring the complete geometry of the objects significantly helps in tracking. By hallucinating unseen regions of objects, we can obtain additional correspondences between the same instance, thus providing robust tracking even under strong change of appearance. From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space. This allows us to derive 6DoF poses for the objects in each frame, along with their correspondence between frames, providing robust object tracking across the RGB-D sequence. Experiments on both synthetic and real-world RGB-D data demonstrate that we achieve state-of-the-art performance on dynamic object tracking. Furthermore, we show that our object completion significantly helps tracking, providing an improvement of 6.5% in mean MOTA.


page 1

page 6

page 7


3D Multi-Object Tracking with Differentiable Pose Estimation

We propose a novel approach for joint 3D multi-object tracking and recon...

Deep 6-DOF Tracking

We present a temporal 6-DOF tracking method which leverages deep learnin...

HDRFusion: HDR SLAM using a low-cost auto-exposure RGB-D sensor

We describe a new method for comparing frame appearance in a frame-to-mo...

HVC-Net: Unifying Homography, Visibility, and Confidence Learning for Planar Object Tracking

Robust and accurate planar tracking over a whole video sequence is vital...

FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image

In this work, we introduce the novel problem of identifying dense canoni...

Robust Multi-Object Tracking by Marginal Inference

Multi-object tracking in videos requires to solve a fundamental problem ...

How to track your dragon: A Multi-Attentional framework for real-time 6-DOF RGB-D Object Pose Tracking

We present a novel multi-attentional convolutional architecture to tack...

Please sign up or login with your details

Forgot password? Click here to reset