In-Hand 3D Object Scanning from an RGB Sequence

by   Shreyas Hampali, et al.

We propose a method for in-hand 3D scanning of an unknown object from a sequence of color images. We cast the problem as reconstructing the object surface from un-posed multi-view images and rely on a neural implicit surface representation that captures both the geometry and the appearance of the object. By contrast with most NeRF-based methods, we do not assume that the camera-object relative poses are known and instead simultaneously optimize both the object shape and the pose trajectory. As global optimization over all the shape and pose parameters is prone to fail without coarse-level initialization of the poses, we propose an incremental approach which starts by splitting the sequence into carefully selected overlapping segments within which the optimization is likely to succeed. We incrementally reconstruct the object shape and track the object poses independently within each segment, and later merge all the segments by aligning poses estimated at the overlapping frames. Finally, we perform a global optimization over all the aligned segments to achieve full reconstruction. We experimentally show that the proposed method is able to reconstruct the shape and color of both textured and challenging texture-less objects, outperforms classical methods that rely only on appearance features, and its performance is close to recent methods that assume known camera poses.


page 1

page 5

page 6

page 7

page 8

page 10

page 11

page 12


Multi-View Neural Surface Reconstruction with Structured Light

Three-dimensional (3D) object reconstruction based on differentiable ren...

LISA: Learning Implicit Shape and Appearance of Hands

This paper proposes a do-it-all neural model of human hands, named LISA....

ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

Autonomous systems need to understand the semantics and geometry of thei...

EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild

We present EMDB, the Electromagnetic Database of Global 3D Human Pose an...

AlignSDF: Pose-Aligned Signed Distance Fields for Hand-Object Reconstruction

Recent work achieved impressive progress towards joint reconstruction of...

Polarization Human Shape and Pose Dataset

Polarization images are known to be able to capture polarized reflected ...

HandNeRF: Neural Radiance Fields for Animatable Interacting Hands

We propose a novel framework to reconstruct accurate appearance and geom...

Please sign up or login with your details

Forgot password? Click here to reset