Few-Shot Keypoint Detection as Task Adaptation via Latent Embeddings

12/09/2021
by   Mel Večerík, et al.
0

Dense object tracking, the ability to localize specific object points with pixel-level accuracy, is an important computer vision task with numerous downstream applications in robotics. Existing approaches either compute dense keypoint embeddings in a single forward pass, meaning the model is trained to track everything at once, or allocate their full capacity to a sparse predefined set of points, trading generality for accuracy. In this paper we explore a middle ground based on the observation that the number of relevant points at a given time are typically relatively few, e.g. grasp points on a target object. Our main contribution is a novel architecture, inspired by few-shot task adaptation, which allows a sparse-style network to condition on a keypoint embedding that indicates which point to track. Our central finding is that this approach provides the generality of dense-embedding models, while offering accuracy significantly closer to sparse-keypoint approaches. We present results illustrating this capacity vs. accuracy trade-off, and demonstrate the ability to zero-shot transfer to new object instances (within-class) using a real-robot pick-and-place task.

READ FULL TEXT

page 1

page 5

page 8

page 9

page 10

page 11

page 12

research
02/18/2022

KINet: Keypoint Interaction Networks for Unsupervised Forward Modeling

Object-centric representation is an essential abstraction for physical r...
research
12/06/2022

PØDA: Prompt-driven Zero-shot Domain Adaptation

Domain adaptation has been vastly investigated in computer vision but st...
research
05/10/2018

Structure-from-Motion using Dense CNN Features with Keypoint Relocalization

Structure from Motion (SfM) using imagery that involves extreme appearan...
research
08/26/2020

Keypoint-Aligned Embeddings for Image Retrieval and Re-identification

Learning embeddings that are invariant to the pose of the object is cruc...
research
05/28/2023

Z-GMOT: Zero-shot Generic Multiple Object Tracking

Despite the significant progress made in recent years, Multi-Object Trac...
research
03/09/2023

KGNv2: Separating Scale and Pose Prediction for Keypoint-based 6-DoF Grasp Synthesis on RGB-D input

We propose a new 6-DoF grasp pose synthesis approach from 2D/2.5D input ...
research
08/03/2023

SpaDen : Sparse and Dense Keypoint Estimation for Real-World Chart Understanding

We introduce a novel bottom-up approach for the extraction of chart data...

Please sign up or login with your details

Forgot password? Click here to reset