3D Multi-Object Tracking with Differentiable Pose Estimation

06/28/2022
by   Dominik Schmauser, et al.
0

We propose a novel approach for joint 3D multi-object tracking and reconstruction from RGB-D sequences in indoor environments. To this end, we detect and reconstruct objects in each frame while predicting dense correspondences mappings into a normalized object space. We leverage those correspondences to inform a graph neural network to solve for the optimal, temporally-consistent 7-DoF pose trajectories of all objects. The novelty of our method is two-fold: first, we propose a new graph-based approach for differentiable pose estimation over time to learn optimal pose trajectories; second, we present a joint formulation of reconstruction and pose estimation along the time axis for robust and geometrically consistent multi-object tracking. In order to validate our approach, we introduce a new synthetic dataset comprising 2381 unique indoor sequences with a total of 60k rendered RGB-D images for multi-object tracking with moving objects and camera positions derived from the synthetic 3D-FRONT dataset. We demonstrate that our method improves the accumulated MOTA score for all test sequences by 24.8 existing state-of-the-art methods. In several ablations on synthetic and real-world sequences, we show that our graph-based, fully end-to-end-learnable approach yields a significant boost in tracking performance.

READ FULL TEXT

page 2

page 8

page 9

page 13

page 15

research
12/15/2020

Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

Multi-object tracking from RGB-D video sequences is a challenging proble...
research
06/23/2022

Unseen Object 6D Pose Estimation: A Benchmark and Baselines

Estimating the 6D pose for unseen objects is in great demand for many re...
research
09/08/2022

PixTrack: Precise 6DoF Object Pose Tracking using NeRF Templates and Feature-metric Alignment

We present PixTrack, a vision based object pose tracking framework using...
research
04/10/2020

End-to-end Learning Improves Static Object Geo-localization in Monocular Video

Accurately estimating the position of static objects, such as traffic li...
research
08/31/2020

Deep Probabilistic Feature-metric Tracking

Dense image alignment from RGB-D images remains a critical issue for rea...
research
12/05/2022

ObjectMatch: Robust Registration using Canonical Object Correspondences

We present ObjectMatch, a semantic and object-centric camera pose estima...
research
05/08/2023

Indoor Localization and Multi-person Tracking Using Privacy Preserving Distributed Camera Network with Edge Computing

Localization of individuals in a built environment is a growing research...

Please sign up or login with your details

Forgot password? Click here to reset