Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images

by   Yigit Baran Can, et al.

We study the problem of estimating 3D shape and pose of an object in terms of keypoints, from a single 2D image. The shape and pose are learned directly from images collected by categories and their partial 2D keypoint annotations.. In this work, we first propose an end-to-end training framework for intermediate 2D keypoints extraction and final 3D shape and pose estimation. The proposed framework is then trained using only the weak supervision of the intermediate 2D keypoints. Additionally, we devise a semi-supervised training framework that benefits from both labeled and unlabeled data. To leverage the unlabeled data, we introduce and exploit the piece-wise planar hull prior of the canonical object shape. These planar hulls are defined manually once per object category, with the help of the keypoints. On the one hand, the proposed method learns to segment these planar hulls from the labeled data. On the other hand, it simultaneously enforces the consistency between predicted keypoints and the segmented hulls on the unlabeled data. The enforced consistency allows us to efficiently use the unlabeled data for the task at hand. The proposed method achieves comparable results with fully supervised state-of-the-art methods by using only half of the annotations. Our source code will be made publicly available.


page 4

page 5

page 6

page 8

page 12

page 13

page 14

page 15


Manifold Graph with Learned Prototypes for Semi-Supervised Image Classification

Recent advances in semi-supervised learning methods rely on estimating c...

SemiMultiPose: A Semi-supervised Multi-animal Pose Estimation Framework

Multi-animal pose estimation is essential for studying animals' social b...

End-to-End Learning of Multi-category 3D Pose and Shape Estimation

In this paper, we study the representation of the shape and pose of obje...

Semi-supervised 3D Hand-Object Pose Estimation via Pose Dictionary Learning

3D hand-object pose estimation is an important issue to understand the i...

MONET: Multiview Semi-supervised Keypoint via Epipolar Divergence

This paper presents MONET---an end-to-end semi-supervised learning frame...

Learning Articulated Shape with Keypoint Pseudo-labels from Web Images

This paper shows that it is possible to learn models for monocular 3D re...

Learning Pose Specific Representations by Predicting Different Views

The labeled data required to learn pose estimation for articulated objec...

Please sign up or login with your details

Forgot password? Click here to reset