Single-Camera 3D Head Fitting for Mixed Reality Clinical Applications

by   Tejas Mane, et al.

We address the problem of estimating the shape of a person's head, defined as the geometry of the complete head surface, from a video taken with a single moving camera, and determining the alignment of the fitted 3D head for all video frames, irrespective of the person's pose. 3D head reconstructions commonly tend to focus on perfecting the face reconstruction, leaving the scalp to a statistical approximation. Our goal is to reconstruct the head model of each person to enable future mixed reality applications. To do this, we recover a dense 3D reconstruction and camera information via structure-from-motion and multi-view stereo. These are then used in a new two-stage fitting process to recover the 3D head shape by iteratively fitting a 3D morphable model of the head with the dense reconstruction in canonical space and fitting it to each person's head, using both traditional facial landmarks and scalp features extracted from the head's segmentation mask. Our approach recovers consistent geometry for varying head shapes, from videos taken by different people, with different smartphones, and in a variety of environments from living rooms to outdoor spaces.


page 2

page 3

page 4

page 6

page 7

page 8


DAD-3DHeads: A Large-scale Dense, Accurate and Diverse Dataset for 3D Head Alignment from a Single Image

We present DAD-3DHeads, a dense and diverse large-scale dataset, and a r...

Head Reconstruction from Internet Photos

3D face reconstruction from Internet photos has recently produced exciti...

Egocentric Field-of-View Localization Using First-Person Point-of-View Devices

We present a technique that uses images, videos and sensor data taken fr...

Instant Multi-View Head Capture through Learnable Registration

Existing methods for capturing datasets of 3D heads in dense semantic co...

Learning to Reconstruct People in Clothing from a Single RGB Camera

We present a learning-based model to infer the personalized 3D shape of ...

A method for automatic forensic facial reconstruction based on dense statistics of soft tissue thickness

In this paper, we present a method for automated estimation of a human f...

Automatic 3D modelling of craniofacial form

Three-dimensional models of craniofacial variation over the general popu...

Please sign up or login with your details

Forgot password? Click here to reset