Weakly-Supervised End-to-End CAD Retrieval to Scan Objects

03/24/2022
by   tim-beyer, et al.
0

CAD model retrieval to real-world scene observations has shown strong promise as a basis for 3D perception of objects and a clean, lightweight mesh-based scene representation; however, current approaches to retrieve CAD models to a query scan rely on expensive manual annotations of 1:1 associations of CAD-scan objects, which typically contain strong lower-level geometric differences. We thus propose a new weakly-supervised approach to retrieve semantically and structurally similar CAD models to a query 3D scanned scene without requiring any CAD-scan associations, and only object detection information as oriented bounding boxes. Our approach leverages a fully-differentiable top-k retrieval layer, enabling end-to-end training guided by geometric and perceptual similarity of the top retrieved CAD models to the scan queries. We demonstrate that our weakly-supervised approach can outperform fully-supervised retrieval methods on challenging real-world ScanNet scans, and maintain robustness for unseen class categories, achieving significantly improved performance over fully-supervised state of the art in zero-shot CAD retrieval.

READ FULL TEXT

page 1

page 10

page 12

page 13

page 22

page 23

research
06/10/2019

End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scans

We present a novel, end-to-end approach to align CAD models to an 3D sca...
research
08/19/2019

Joint Embedding of 3D Scan and CAD Objects

3D scan geometry and CAD models often contain complementary information ...
research
12/03/2021

ROCA: Robust CAD Model Retrieval and Alignment from a Single Image

We present ROCA, a novel end-to-end approach that retrieves and aligns 3...
research
12/22/2022

Automatically Annotating Indoor Images with CAD Models via RGB-D Scans

We present an automatic method for annotating images of indoor scenes wi...
research
02/11/2022

Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer

Text spotting end-to-end methods have recently gained attention in the l...
research
03/27/2023

What Can Human Sketches Do for Object Detection?

Sketches are highly expressive, inherently capturing subjective and fine...
research
02/15/2022

Deep-QPP: A Pairwise Interaction-based Deep Learning Model for Supervised Query Performance Prediction

Motivated by the recent success of end-to-end deep neural models for ran...

Please sign up or login with your details

Forgot password? Click here to reset