NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation

03/09/2022
by   Fu Li, et al.
26

Pose estimation of 3D objects in monocular images is a fundamental and long-standing problem in computer vision. Existing deep learning approaches for 6D pose estimation typically rely on the assumption of availability of 3D object models and 6D pose annotations. However, precise annotation of 6D poses in real data is intricate, time-consuming and not scalable, while synthetic data scales well but lacks realism. To avoid these problems, we present a weakly-supervised reconstruction-based pipeline, named NeRF-Pose, which needs only 2D object segmentation and known relative camera poses during training. Following the first-reconstruct-then-regress idea, we first reconstruct the objects from multiple views in the form of an implicit neural representation. Then, we train a pose regression network to predict pixel-wise 2D-3D correspondences between images and the reconstructed model. At inference, the approach only needs a single image as input. A NeRF-enabled PnP+RANSAC algorithm is used to estimate stable and accurate pose from the predicted correspondences. Experiments on LineMod and LineMod-Occlusion show that the proposed method has state-of-the-art accuracy in comparison to the best 6D pose estimation methods in spite of being trained only with weak labels. Besides, we extend the Homebrewed DB dataset with more real training images to support the weakly supervised task and achieve compelling results on this dataset. The extended dataset and code will be released soon.

READ FULL TEXT

page 1

page 3

page 6

page 9

research
05/17/2018

It's all Relative: Monocular 3D Human Pose Estimation from Weakly Supervised Data

We address the problem of 3D human pose estimation from 2D input images ...
research
03/07/2022

Weakly Supervised Learning of Keypoints for 6D Object Pose Estimation

State-of-the-art approaches for 6D object pose estimation require large ...
research
11/20/2015

Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning

We propose a method for hand pose estimation based on a deep regressor t...
research
03/02/2022

3D object reconstruction and 6D-pose estimation from 2D shape for robotic grasping of objects

We propose a method for 3D object reconstruction and 6D-pose estimation ...
research
08/22/2018

Can 3D Pose be Learned from 2D Projections Alone?

3D pose estimation from a single image is a challenging task in computer...
research
08/07/2023

A Horse with no Labels: Self-Supervised Horse Pose Estimation from Unlabelled Images and Synthetic Prior

Obtaining labelled data to train deep learning methods for estimating an...
research
06/14/2020

PrimA6D: Rotational Primitive Reconstruction for Enhanced and Robust 6D Pose Estimation

In this paper, we introduce a rotational primitive prediction based 6D o...

Please sign up or login with your details

Forgot password? Click here to reset