PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal

05/26/2021
by   Si Liu, et al.
7

In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively. Existing methods have achieved much advancement in constrained scenarios, but it is still very challenging for them to transfer makeup between images with large pose and expression differences, or handle makeup details like blush on cheeks or highlight on the nose. In addition, they are hardly able to control the degree of makeup during transferring or to transfer a specified part in the input face. In this work, we propose the PSGAN++, which is capable of performing both detail-preserving makeup transfer and effective makeup removal. For makeup transfer, PSGAN++ uses a Makeup Distill Network to extract makeup information, which is embedded into spatial-aware makeup matrices. We also devise an Attentive Makeup Morphing module that specifies how the makeup in the source image is morphed from the reference image, and a makeup detail loss to supervise the model within the selected makeup detail area. On the other hand, for makeup removal, PSGAN++ applies an Identity Distill Network to embed the identity information from with-makeup images into identity matrices. Finally, the obtained makeup/identity matrices are fed to a Style Transfer Network that is able to edit the feature maps to achieve makeup transfer or removal. To evaluate the effectiveness of our PSGAN++, we collect a Makeup Transfer In the Wild dataset that contains images with diverse poses and expressions and a Makeup Transfer High-Resolution dataset that contains high-resolution images. Experiments demonstrate that PSGAN++ not only achieves state-of-the-art results with fine makeup details even in cases of large pose/expression differences but also can perform partial or degree-controllable makeup transfer.

READ FULL TEXT

page 2

page 7

page 9

page 10

page 11

page 12

page 13

page 14

09/16/2019

PSGAN: Pose-Robust Spatial-Aware GAN for Customizable Makeup Transfer

We propose a novel Pose-robust Spatial-aware GAN (PSGAN) for transferrin...
03/27/2020

Local Facial Makeup Transfer via Disentangled Representation

Facial makeup transfer aims to render a non-makeup face image in an arbi...
03/02/2022

Styleverse: Towards Identity Stylization across Heterogeneous Domains

We propose a new challenging task namely IDentity Stylization (IDS) acro...
04/21/2021

SOGAN: 3D-Aware Shadow and Occlusion Robust GAN for Makeup Transfer

In recent years, virtual makeup applications have become more and more p...
05/26/2022

One-Shot Face Reenactment on Megapixels

The goal of face reenactment is to transfer a target expression and head...
12/07/2021

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

Makeup transfer is not only to extract the makeup style of the reference...
07/20/2022

M2-Net: Multi-stages Specular Highlight Detection and Removal in Multi-scenes

In this paper, we propose a novel uniformity framework for highlight det...