High Fidelity Face Manipulation with Extreme Pose and Expression

by   Chaoyou Fu, et al.

Face manipulation has shown remarkable advances with the flourish of Generative Adversarial Networks. However, due to the difficulties of controlling the structure and texture in high-resolution, it is challenging to simultaneously model pose and expression during manipulation. In this paper, we propose a novel framework that simplifies face manipulation with extreme pose and expression into two correlated stages: a boundary prediction stage and a disentangled face synthesis stage. In the first stage, we propose to use a boundary image for joint pose and expression modeling. An encoder-decoder network is employed to predict the boundary image of the target face in a semi-supervised way. Pose and expression estimators are used to improve the prediction accuracy. In the second stage, the predicted boundary image and the original face are encoded into the structure and texture latent space by two encoder networks respectively. A proxy network and a feature threshold loss are further imposed as constraints to disentangle the latent space. In addition, we build up a new high quality Multi-View Face (MVF-HQ) database that contains 120K high-resolution face images of 479 identities with pose and expression variations, which will be released soon. Qualitative and quantitative experiments on four databases show that our method pushes forward the advance of extreme face manipulation from 128 × 128 resolution to 1024 × 1024 resolution, and significantly improves the face recognition performance under large poses.


page 5

page 6

page 12

page 13

page 14

page 15

page 16

page 17


MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation

Recent advances in generative adversarial networks (GANs) have led to re...

Make a Face: Towards Arbitrary High Fidelity Face Manipulation

Recent studies have shown remarkable success in face manipulation task w...

Video2StyleGAN: Encoding Video in Latent Space for Manipulation

Many recent works have been proposed for face image editing by leveragin...

PI-GAN: Learning Pose Independent representations for multiple profile face synthesis

Generating a pose-invariant representation capable of synthesizing multi...

OSTeC: One-Shot Texture Completion

The last few years have witnessed the great success of non-linear genera...

3D Guided Fine-Grained Face Manipulation

We present a method for fine-grained face manipulation. Given a face ima...

Pose Agnostic Cross-spectral Hallucination via Disentangling Independent Factors

The cross-sensor gap is one of the challenges that arise much research i...

Please sign up or login with your details

Forgot password? Click here to reset