Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition

11/20/2018
by   Lei Li, et al.
0

Freehand sketching is a dynamic process where points are sequentially sampled and grouped as strokes for sketch acquisition on electronic devices. To recognize a sketched object, most existing methods discard such important temporal ordering and grouping information from human and simply rasterize sketches into binary images for classification. In this paper, we propose a novel single-branch attentive network architecture RNN-Rasterization-CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively. To bridge the gap between these two spaces in neural networks, we propose a neural line rasterization module to convert the vector sketch along with the attention estimated by RNN into a bitmap image, which is subsequently consumed by CNN. The neural line rasterization module is designed in a differentiable way to yield a unified pipeline for end-to-end learning. We perform experiments on existing large-scale sketch recognition benchmarks and show that by exploiting the sketch dynamics with the attention mechanism, our method is more robust and achieves better performance than the state-of-the-art methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/11/2017

A Neural Representation of Sketch Drawings

We present sketch-rnn, a recurrent neural network (RNN) able to construc...
research
04/04/2018

SketchMate: Deep Hashing for Million-Scale Human Sketch Retrieval

We propose a deep hashing framework for sketch retrieval that, for the f...
research
08/26/2021

SketchLattice: Latticed Representation for Sketch Manipulation

The key challenge in designing a sketch representation lies with handlin...
research
11/09/2020

Sketch-Inspector: a Deep Mixture Model for High-Quality Sketch Generation of Cats

With the involvement of artificial intelligence (AI), sketches can be au...
research
08/09/2017

Sequential Dual Deep Learning with Shape and Texture Features for Sketch Recognition

Recognizing freehand sketches with high arbitrariness is greatly challen...
research
09/13/2017

Sketch-pix2seq: a Model to Generate Sketches of Multiple Categories

Sketch is an important media for human to communicate ideas, which refle...
research
08/03/2016

Language free character recognition using character sketch and center of gravity shifting

In this research, we present a heuristic method for character recognitio...

Please sign up or login with your details

Forgot password? Click here to reset