Fast Sketch Segmentation and Labeling with Deep Learning

by   Lei Li, et al.

We present a simple and efficient method based on deep learning to automatically decompose sketched objects into semantically valid parts. We train a deep neural network to transfer existing segmentations and labelings from 3D models to freehand sketches without requiring numerous well-annotated sketches as training data. The network takes the binary image of a sketched object as input and produces a corresponding segmentation map with per-pixel labelings as output. A subsequent post-process procedure with multi-label graph cuts further refines the segmentation and labeling result. We validate our proposed method on two sketch datasets. Experiments show that our method outperforms the state-of-the-art method in terms of segmentation and labeling accuracy and is significantly faster, enabling further integration in interactive drawing systems. We demonstrate the efficiency of our method in a sketch-based modeling application that automatically transforms input sketches into 3D models by part assembly.


page 1

page 2

page 4

page 5

page 7

page 8

page 10

page 11


SFSegNet: Parse Freehand Sketches using Deep Fully Convolutional Networks

Parsing sketches via semantic segmentation is attractive but challenging...

SketchGCN: Semantic Sketch Segmentation with Graph Convolutional Networks

We introduce SketchGCN, a graph convolutional neural network for semanti...

DeepSketchHair: Deep Sketch-based 3D Hair Modeling

We present sketchhair, a deep learning based tool for interactive modeli...

One Sketch for All: One-Shot Personalized Sketch Segmentation

We present the first one-shot personalized sketch segmentation method. W...

Neural Network Based Reconstruction of a 3D Object from a 2D Wireframe

We propose a new approach for constructing a 3D representation from a 2D...

Mastering Sketching: Adversarial Augmentation for Structured Prediction

We present an integral framework for training sketch simplification netw...

TreeSketchNet: From Sketch To 3D Tree Parameters Generation

3D modeling of non-linear objects from stylized sketches is a challenge ...