Mastering Sketching: Adversarial Augmentation for Structured Prediction

03/27/2017
by   Edgar Simo-Serra, et al.
0

We present an integral framework for training sketch simplification networks that convert challenging rough sketches into clean line drawings. Our approach augments a simplification network with a discriminator network, training both networks jointly so that the discriminator network discerns whether a line drawing is a real training data or the output of the simplification network, which in turn tries to fool it. This approach has two major advantages. First, because the discriminator network learns the structure in line drawings, it encourages the output sketches of the simplification network to be more similar in appearance to the training sketches. Second, we can also train the simplification network with additional unsupervised data, using the discriminator network as a substitute teacher. Thus, by adding only rough sketches without simplified line drawings, or only line drawings without the original rough sketches, we can improve the quality of the sketch simplification. We show how our framework can be used to train models that significantly outperform the state of the art in the sketch simplification task, despite using the same architecture for inference. We additionally present an approach to optimize for a single image, which improves accuracy at the cost of additional computation time. Finally, we show that, using the same framework, it is possible to train the network to perform the inverse problem, i.e., convert simple line sketches into pencil drawings, which is not possible using the standard mean squared error loss. We validate our framework with two user tests, where our approach is preferred to the state of the art in sketch simplification 92.3 5.

READ FULL TEXT
research
02/08/2017

An Adversarial Regularisation for Semi-Supervised Training of Structured Output Neural Networks

We propose a method for semi-supervised training of structured-output ne...
research
10/12/2018

Unsupervised Facial Geometry Learning for Sketch to Photo Synthesis

Face sketch-photo synthesis is a critical application in law enforcement...
research
10/14/2019

Sketch-Specific Data Augmentation for Freehand Sketch Recognition

Sketch recognition remains a significant challenge due to the limited tr...
research
05/24/2018

Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation

Random data augmentation is a critical technique to avoid overfitting in...
research
07/31/2018

Fast Sketch Segmentation and Labeling with Deep Learning

We present a simple and efficient method based on deep learning to autom...
research
11/09/2020

Sketch-Inspector: a Deep Mixture Model for High-Quality Sketch Generation of Cats

With the involvement of artificial intelligence (AI), sketches can be au...
research
01/10/2018

Unsupervised Despeckling

Contrast and quality of ultrasound images are adversely affected by the ...

Please sign up or login with your details

Forgot password? Click here to reset