Deep Generation of Face Images from Sketches

by   Shu-Yu Chen, et al.

Recent deep image-to-image translation techniques allow fast generation of face images from freehand sketches. However, existing solutions tend to overfit to sketches, thus requiring professional sketches or even edge maps as input. To address this issue, our key idea is to implicitly model the shape space of plausible face images and synthesize a face image in this space to approximate an input sketch. We take a local-to-global approach. We first learn feature embeddings of key face components, and push corresponding parts of input sketches towards underlying component manifolds defined by the feature vectors of face component samples. We also propose another deep neural network to learn the mapping from the embedded component features to realistic images with multi-channel feature maps as intermediate results to improve the information flow. Our method essentially uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches. Our tool is easy to use even for non-artists, while still supporting fine-grained control of shape details. Both qualitative and quantitative evaluations show the superior generation ability of our system to existing and alternative solutions. The usability and expressiveness of our system are confirmed by a user study.


page 1

page 8

page 9

page 10

page 11

page 12

page 14

page 15


DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN

The research topic of sketch-to-portrait generation has witnessed a boos...

DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control

Recent facial image synthesis methods have been mainly based on conditio...

Effect of Instance Normalization on Fine-Grained Control for Sketch-Based Face Image Generation

Sketching is an intuitive and effective way for content creation. While ...

DeepFacePencil: Creating Face Images from Freehand Sketches

In this paper, we explore the task of generating photo-realistic face im...

Convolutional Sketch Inversion

In this paper, we use deep neural networks for inverting face sketches t...

Learning to Hallucinate Face Images via Component Generation and Enhancement

We propose a two-stage method for face hallucination. First, we generate...

FaceCook: Face Generation Based on Linear Scaling Factors

With the excellent disentanglement properties of state-of-the-art genera...