Deep Facial Synthesis: A New Challenge

by   Deng-Ping Fan, et al.

The goal of this paper is to conduct a comprehensive study on the facial sketch synthesis (FSS) problem. However, due to the high costs in obtaining hand-drawn sketch datasets, there lacks a complete benchmark for assessing the development of FSS algorithms over the last decade. As such, we first introduce a high-quality dataset for FSS, named FS2K, which consists of 2,104 image-sketch pairs spanning three types of sketch styles, image backgrounds, lighting conditions, skin colors, and facial attributes. FS2K differs from previous FSS datasets in difficulty, diversity, and scalability, and should thus facilitate the progress of FSS research. Second, we present the largest-scale FSS study by investigating 139 classical methods, including 24 handcrafted feature based facial sketch synthesis approaches, 37 general neural-style transfer methods, 43 deep image-to-image translation methods, and 35 image-to-sketch approaches. Besides, we elaborate comprehensive experiments for existing 19 cutting-edge models. Third, we present a simple baseline for FSS, named FSGAN. With only two straightforward components, i.e., facial-aware masking and style-vector expansion, FSGAN surpasses the performance of all previous state-of-the-art models on the proposed FS2K dataset, by a large margin. Finally, we conclude with lessons learned over the past years, and point out several unsolved challenges. Our open-source code is available at


page 2

page 3

page 11

page 12

page 16

page 17

page 18

page 20


Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation

Facial sketch synthesis (FSS) aims to generate a vivid sketch portrait f...

Face Sketch Synthesis with Style Transfer using Pyramid Column Feature

In this paper, we propose a novel framework based on deep neural network...

Learning to Sketch Human Facial Portraits using Personal Styles by Case-Based Reasoning

This paper employs case-based reasoning (CBR) to capture the personal st...

MOST-Net: A Memory Oriented Style Transfer Network for Face Sketch Synthesis

Face sketch synthesis has been widely used in multi-media entertainment ...

Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark

Face sketch generation has attracted much attention in the field of visu...

Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation

Facial landmark detection, or face alignment, is a fundamental task that...

DifferSketching: How Differently Do People Sketch 3D Objects?

Multiple sketch datasets have been proposed to understand how people dra...

Please sign up or login with your details

Forgot password? Click here to reset