OSTeC: One-Shot Texture Completion

12/30/2020
by   Baris Gecer, et al.
6

The last few years have witnessed the great success of non-linear generative models in synthesizing high-quality photorealistic face images. Many recent 3D facial texture reconstruction and pose manipulation from a single image approaches still rely on large and clean face datasets to train image-to-image Generative Adversarial Networks (GANs). Yet the collection of such a large scale high-resolution 3D texture dataset is still very costly and difficult to maintain age/ethnicity balance. Moreover, regression-based approaches suffer from generalization to the in-the-wild conditions and are unable to fine-tune to a target-image. In this work, we propose an unsupervised approach for one-shot 3D facial texture completion that does not require large-scale texture datasets, but rather harnesses the knowledge stored in 2D face generators. The proposed approach rotates an input image in 3D and fill-in the unseen regions by reconstructing the rotated image in a 2D face generator, based on the visible parts. Finally, we stitch the most visible textures at different angles in the UV image-plane. Further, we frontalize the target image by projecting the completed texture into the generator. The qualitative and quantitative experiments demonstrate that the completed UV textures and frontalized images are of high quality, resembles the original identity, can be used to train a texture GAN model for 3DMM fitting and improve pose-invariant face recognition.

READ FULL TEXT

page 1

page 8

page 13

page 14

page 15

page 17

page 18

page 19

05/16/2021

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

A lot of work has been done towards reconstructing the 3D facial structu...
02/15/2019

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

In the past few years, a lot of work has been done towards reconstructin...
08/24/2018

High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networks

In the past several decades, many attempts have been made to model synth...
10/19/2018

MsCGAN: Multi-scale Conditional Generative Adversarial Networks for Person Image Generation

To synthesize high quality person images with arbitrary poses is challen...
11/25/2020

StyleUV: Diverse and High-fidelity UV Map Generative Model

Reconstructing 3D human faces in the wild with the 3D Morphable Model (3...
04/29/2019

TileGAN: Synthesis of Large-Scale Non-Homogeneous Textures

We tackle the problem of texture synthesis in the setting where many inp...
02/10/2019

Cross-spectral Face Completion for NIR-VIS Heterogeneous Face Recognition

Near infrared-visible (NIR-VIS) heterogeneous face recognition refers to...