Few-shot Geometry-Aware Keypoint Localization

by   Xingzhe He, et al.

Supervised keypoint localization methods rely on large manually labeled image datasets, where objects can deform, articulate, or occlude. However, creating such large keypoint labels is time-consuming and costly, and is often error-prone due to inconsistent labeling. Thus, we desire an approach that can learn keypoint localization with fewer yet consistently annotated images. To this end, we present a novel formulation that learns to localize semantically consistent keypoint definitions, even for occluded regions, for varying object categories. We use a few user-labeled 2D images as input examples, which are extended via self-supervision using a larger unlabeled dataset. Unlike unsupervised methods, the few-shot images act as semantic shape constraints for object localization. Furthermore, we introduce 3D geometry-aware constraints to uplift keypoints, achieving more accurate 2D localization. Our general-purpose formulation paves the way for semantically conditioned generative modeling and attains competitive or state-of-the-art accuracy on several datasets, including human faces, eyes, animals, cars, and never-before-seen mouth interior (teeth) localization tasks, not attempted by the previous few-shot methods. Project page: https://xingzhehe.github.io/FewShot3DKP/https://xingzhehe.github.io/FewShot3DKP/


page 1

page 4

page 6

page 8

page 14


Semi-supervised Keypoint Localization

Knowledge about the locations of keypoints of an object in an image can ...

Unsupervised Volumetric Animation

We propose a novel approach for unsupervised 3D animation of non-rigid d...

Self-Supervised Viewpoint Learning From Image Collections

Training deep neural networks to estimate the viewpoint of objects requi...

Learning Articulated Shape with Keypoint Pseudo-labels from Web Images

This paper shows that it is possible to learn models for monocular 3D re...

Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species

Current non-rigid object keypoint detectors perform well on a chosen kin...

Long-Lived Accurate Keypoints in Event Streams

We present a novel end-to-end approach to keypoint detection and trackin...

CoKe: Localized Contrastive Learning for Robust Keypoint Detection

Today's most popular approaches to keypoint detection learn a holistic r...

Please sign up or login with your details

Forgot password? Click here to reset