SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

12/07/2021
by   Zhaoyang Sun, et al.
0

Makeup transfer is not only to extract the makeup style of the reference image, but also to render the makeup style to the semantic corresponding position of the target image. However, most existing methods focus on the former and ignore the latter, resulting in a failure to achieve desired results. To solve the above problems, we propose a unified Symmetric Semantic-Aware Transformer (SSAT) network, which incorporates semantic correspondence learning to realize makeup transfer and removal simultaneously. In SSAT, a novel Symmetric Semantic Corresponding Feature Transfer (SSCFT) module and a weakly supervised semantic loss are proposed to model and facilitate the establishment of accurate semantic correspondence. In the generation process, the extracted makeup features are spatially distorted by SSCFT to achieve semantic alignment with the target image, then the distorted makeup features are combined with unmodified makeup irrelevant features to produce the final result. Experiments show that our method obtains more visually accurate makeup transfer results, and user study in comparison with other state-of-the-art makeup transfer methods reflects the superiority of our method. Besides, we verify the robustness of the proposed method in the difference of expression and pose, object occlusion scenes, and extend it to video makeup transfer. Code will be available at https://gitee.com/sunzhaoyang0304/ssat-msp.

READ FULL TEXT

page 1

page 2

page 5

page 7

page 11

page 12

page 13

page 14

research
03/04/2022

Semi-parametric Makeup Transfer via Semantic-aware Correspondence

The large discrepancy between the source non-makeup image and the refere...
research
02/27/2023

Self Correspondence Distillation for End-to-End Weakly-Supervised Semantic Segmentation

Efficiently training accurate deep models for weakly supervised semantic...
research
09/16/2019

PSGAN: Pose-Robust Spatial-Aware GAN for Customizable Makeup Transfer

We propose a novel Pose-robust Spatial-aware GAN (PSGAN) for transferrin...
research
07/03/2020

Anatomy-Aware Siamese Network: Exploiting Semantic Asymmetry for Accurate Pelvic Fracture Detection in X-ray Images

Visual cues of enforcing bilaterally symmetric anatomies as normal findi...
research
07/19/2023

AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks

To deliver the artistic expression of the target style, recent studies e...
research
12/02/2021

Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization

Exemplar-based colorization approaches rely on reference image to provid...
research
12/12/2022

BeautyREC: Robust, Efficient, and Content-preserving Makeup Transfer

In this work, we propose a Robust, Efficient, and Component-specific mak...

Please sign up or login with your details

Forgot password? Click here to reset