3D human tongue reconstruction from single "in-the-wild" images

06/23/2021
by   Stylianos Ploumpis, et al.
10

3D face reconstruction from a single image is a task that has garnered increased interest in the Computer Vision community, especially due to its broad use in a number of applications such as realistic 3D avatar creation, pose invariant face recognition and face hallucination. Since the introduction of the 3D Morphable Model in the late 90's, we witnessed an explosion of research aiming at particularly tackling this task. Nevertheless, despite the increasing level of detail in the 3D face reconstructions from single images mainly attributed to deep learning advances, finer and highly deformable components of the face such as the tongue are still absent from all 3D face models in the literature, although being very important for the realness of the 3D avatar representations. In this work we present the first, to the best of our knowledge, end-to-end trainable pipeline that accurately reconstructs the 3D face together with the tongue. Moreover, we make this pipeline robust in "in-the-wild" images by introducing a novel GAN method tailored for 3D tongue surface generation. Finally, we make publicly available to the community the first diverse tongue dataset, consisting of 1,800 raw scans of 700 individuals varying in gender, age, and ethnicity backgrounds. As we demonstrate in an extensive series of quantitative as well as qualitative experiments, our model proves to be robust and realistically captures the 3D tongue structure, even in adverse "in-the-wild" conditions.

READ FULL TEXT

page 1

page 3

page 4

page 7

page 8

research
03/30/2020

AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"

Over the last years, with the advent of Generative Adversarial Networks ...
research
11/13/2018

Pose Invariant 3D Face Reconstruction

3D face reconstruction is an important task in the field of computer vis...
research
03/31/2017

End-To-End Face Detection and Recognition

Plenty of face detection and recognition methods have been proposed and ...
research
10/07/2020

A Human Ear Reconstruction Autoencoder

The ear, as an important part of the human head, has received much less ...
research
06/01/2016

A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

Face analysis techniques have become a crucial component of human-machin...
research
03/14/2018

Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild

This paper investigates the evaluation of dense 3D face reconstruction f...
research
05/01/2019

3DFaceGAN: Adversarial Nets for 3D Face Representation, Generation, and Translation

Over the past few years, Generative Adversarial Networks (GANs) have gar...

Please sign up or login with your details

Forgot password? Click here to reset