PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

09/15/2022
by   Van Nguyen Nguyen, et al.
7

Estimating the relative pose of a new object without prior knowledge is a hard problem, while it is an ability very much needed in robotics and Augmented Reality. We present a method for tracking the 6D motion of objects in RGB video sequences when neither the training images nor the 3D geometry of the objects are available. In contrast to previous works, our method can therefore consider unknown objects in open world instantly, without requiring any prior information or a specific training phase. We consider two architectures, one based on two frames, and the other relying on a Transformer Encoder, which can exploit an arbitrary number of past frames. We train our architectures using only synthetic renderings with domain randomization. Our results on challenging datasets are on par with previous works that require much more information (training images of the target objects, 3D models, and/or depth data). Our source code is available at https://github.com/nv-nguyen/pizza

READ FULL TEXT

page 1

page 3

page 7

page 8

page 13

page 14

page 15

page 16

research
03/31/2022

Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions

We present a method that can recognize new objects and estimate their 3D...
research
06/08/2020

Multimodal Future Localization and Emergence Prediction for Objects in Egocentric View with a Reachability Prior

In this paper, we investigate the problem of anticipating future dynamic...
research
05/13/2022

KG-SP: Knowledge Guided Simple Primitives for Open World Compositional Zero-Shot Learning

The goal of open-world compositional zero-shot learning (OW-CZSL) is to ...
research
08/10/2023

Follow Anything: Open-set detection, tracking, and following in real-time

Tracking and following objects of interest is critical to several roboti...
research
10/03/2022

SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB image

Estimating 3D shapes and poses of static objects from a single image has...
research
05/29/2021

Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains

Tracking the 6D pose of objects in video sequences is important for robo...
research
07/20/2023

CNOS: A Strong Baseline for CAD-based Novel Object Segmentation

We propose a simple three-stage approach to segment unseen objects in RG...

Please sign up or login with your details

Forgot password? Click here to reset