DeepAI AI Chat
Log In Sign Up

Towards Harmonized Regional Style Transfer and Manipulation for Facial Images

by   Cong Wang, et al.

Regional facial image synthesis conditioned on semantic mask has achieved great success using generative adversarial networks. However, the appearance of different regions may be inconsistent with each other when conducting regional image editing. In this paper, we focus on the problem of harmonized regional style transfer and manipulation for facial images. The proposed approach supports regional style transfer and manipulation at the same time. A multi-scale encoder and style mapping networks are proposed in our work. The encoder is responsible for extracting regional styles of real faces. Style mapping networks generate styles from random samples for all facial regions. As the key part of our work, we propose a multi-region style attention module to adapt the multiple regional style embeddings from a reference image to a target image for generating harmonious and plausible results. Furthermore, we propose a new metric "harmony score" and conduct experiments in a challenging setting: three widely used face datasets are involved and we test the model by transferring the regional facial appearance between datasets. Images in different datasets are usually quite different, which makes the inconsistency between target and reference regions more obvious. Results show that our model can generate reliable style transfer and multi-modal manipulation results compared with SOTAs. Furthermore, we show two face editing applications using the proposed approach.


page 1

page 2

page 6

page 8

page 9

page 11

page 12

page 13


Style Mixer: Semantic-aware Multi-Style Transfer Network

Recent neural style transfer frameworks have obtained astonishing visual...

Any-to-Any Style Transfer: Making Picasso and Da Vinci Collaborate

Style transfer aims to render the style of a given image for style refer...

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer

Style transfer describes the rendering of an image semantic content as d...

Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer

Style transfer of polyphonic music recordings is a challenging task when...

Massive Styles Transfer with Limited Labeled Data

Language style transfer has attracted more and more attention in the pas...

Style Aggregated Network for Facial Landmark Detection

Recent advances in facial landmark detection achieve success by learning...

MakeupBag: Disentangling Makeup Extraction and Application

This paper introduces MakeupBag, a novel method for automatic makeup sty...