MarioNette: Self-Supervised Sprite Learning

by   Dmitriy Smirnov, et al.

Visual content often contains recurring elements. Text is made up of glyphs from the same font, animations, such as cartoons or video games, are composed of sprites moving around the screen, and natural videos frequently have repeated views of objects. In this paper, we propose a deep learning approach for obtaining a graphically disentangled representation of recurring elements in a completely self-supervised manner. By jointly learning a dictionary of texture patches and training a network that places them onto a canvas, we effectively deconstruct sprite-based content into a sparse, consistent, and interpretable representation that can be easily used in downstream tasks. Our framework offers a promising approach for discovering recurring patterns in image collections without supervision.


page 1

page 2

page 3

page 4

page 6

page 7

page 9

page 10


Demystifying Self-Supervised Learning: An Information-Theoretical Framework

Self-supervised representation learning adopts self-defined signals as s...

Self-supervised Co-training for Video Representation Learning

The objective of this paper is visual-only self-supervised video represe...

iBoot: Image-bootstrapped Self-Supervised Video Representation Learning

Learning visual representations through self-supervision is an extremely...

Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks

Self-supervised learning aims to learn image feature representations wit...

Self-Supervised Metric Learning in Multi-View Data: A Downstream Task Perspective

Self-supervised metric learning has been a successful approach for learn...

SCOPS: Self-Supervised Co-Part Segmentation

Parts provide a good intermediate representation of objects that is robu...

Quantitative analysis of visual representation of sign elements in COVID-19 context

Representation is the way in which human beings re-present the reality o...