From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion

04/08/2022
by   Weiguang Zhao, et al.
0

We consider the problem of Multi-view 3D Face Reconstruction (MVR) with weakly supervised learning that leverages a limited number of 2D face images (e.g. 3) to generate a high-quality 3D face model with very light annotation. Despite their encouraging performance, present MVR methods simply concatenate multi-view image features and pay less attention to critical areas (e.g. eye, brow, nose and mouth). To this end, we propose a novel model called Deep Fusion MVR (DF-MVR) and design a multi-view encoding to a single decoding framework with skip connections, able to extract, integrate, and compensate deep features with attention from multi-view images. In addition, we develop a multi-view face parse network to learn, identify, and emphasize the critical common face area. Finally, though our model is trained with a few 2D images, it can reconstruct an accurate 3D model even if one single 2D image is input. We conduct extensive experiments to evaluate various multi-view 3D face reconstruction methods. Our proposed model attains superior performance, leading to 11.4 MVRs. Source codes are available in the supplementary materials.

READ FULL TEXT

page 3

page 4

page 5

page 6

page 7

page 10

page 11

page 12

research
08/09/2017

Deep Face Feature for Face Alignment and Reconstruction

In this paper, we propose a novel face feature extraction method based o...
research
03/20/2019

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set

Recently, deep learning based 3D face reconstruction methods have shown ...
research
04/19/2023

ASM: Adaptive Skinning Model for High-Quality 3D Face Modeling

The research fields of parametric face models and 3D face reconstruction...
research
06/23/2021

LegoFormer: Transformers for Block-by-Block Multi-view 3D Reconstruction

Most modern deep learning-based multi-view 3D reconstruction techniques ...
research
08/05/2021

Semi- and Self-Supervised Multi-View Fusion of 3D Microscopy Images using Generative Adversarial Networks

Recent developments in fluorescence microscopy allow capturing high-reso...
research
08/15/2023

Multi-view 3D Face Reconstruction Based on Flame

At present, face 3D reconstruction has broad application prospects in va...
research
02/28/2023

A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images

Limited by the nature of the low-dimensional representational capacity o...

Please sign up or login with your details

Forgot password? Click here to reset