High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks

10/27/2017
by   Lidan Wang, et al.
0

Synthesizing face sketches from real photos and its inverse are well studied problems and they have many applications in digital forensics and entertainment. However, photo/sketch synthesis remains a challenging problem due to the fact that photo and sketch have different characteristics. In this work, we consider this task as an image-to-image translation problem and explore the recently popular generative models (GANs) to generate high-quality realistic photos from sketches and sketches from photos. Recent methods such as Pix2Pix, CycleGAN and DualGAN have shown promising results on image-to-image translation problems and photo-to-sketch synthesis in particular, however, they are known to have limited abilities in generating high-resolution realistic images. To this end, we propose a novel synthesis framework called Photo-Sketch Synthesis using Multi-Adversarial Networks, (PS2-MAN) that iteratively generates low resolution to high resolution images in an adversarial way. The hidden layers of the generator are supervised to first generate lower resolution images followed by implicit refinement in the network to generate higher resolution images. Furthermore, since photo-sketch synthesis is a coupled/paired translation problem where photo-sketch and sketch-photo are equally important, we leverage the pair information in the CycleGAN framework. Evaluation of the proposed method is performed on two datasets: CUHK and CUFSF. Both Image Quality Assessment (IQA) and Photo-Sketch Matching experiments are conducted to demonstrate the superior performance of our framework in comparison to existing state-of-the-art solutions. Additionally, ablation studies are conducted to verify the effectiveness iterative synthesis and various loss functions.

READ FULL TEXT

page 1

page 5

page 6

page 7

research
10/23/2021

Face sketch to photo translation using generative adversarial networks

Translating face sketches to photo-realistic faces is an interesting and...
research
10/12/2018

Unsupervised Facial Geometry Learning for Sketch to Photo Synthesis

Face sketch-photo synthesis is a critical application in law enforcement...
research
03/30/2021

Identity-Aware CycleGAN for Face Photo-Sketch Synthesis and Recognition

Face photo-sketch synthesis and recognition has many applications in dig...
research
11/01/2019

Cali-Sketch: Stroke Calibration and Completion for High-Quality Face Image Generation from Poorly-Drawn Sketches

Image generation task has received increasing attention because of its w...
research
01/05/2022

Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning

In recent years, significant progress has been achieved in biphasic face...
research
07/18/2023

Semi-supervised Cycle-GAN for face photo-sketch translation in the wild

The performance of face photo-sketch translation has improved a lot than...
research
04/12/2021

Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis

In this paper, we explore the open-domain sketch-to-photo translation, w...

Please sign up or login with your details

Forgot password? Click here to reset