MarioNette: Self-Supervised Sprite Learning

04/29/2021
by   Dmitriy Smirnov, et al.
20

Visual content often contains recurring elements. Text is made up of glyphs from the same font, animations, such as cartoons or video games, are composed of sprites moving around the screen, and natural videos frequently have repeated views of objects. In this paper, we propose a deep learning approach for obtaining a graphically disentangled representation of recurring elements in a completely self-supervised manner. By jointly learning a dictionary of texture patches and training a network that places them onto a canvas, we effectively deconstruct sprite-based content into a sparse, consistent, and interpretable representation that can be easily used in downstream tasks. Our framework offers a promising approach for discovering recurring patterns in image collections without supervision.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 7

page 9

page 10

06/10/2020

Demystifying Self-Supervised Learning: An Information-Theoretical Framework

Self-supervised representation learning adopts self-defined signals as s...
10/19/2020

Self-supervised Co-training for Video Representation Learning

The objective of this paper is visual-only self-supervised video represe...
06/16/2022

iBoot: Image-bootstrapped Self-Supervised Video Representation Learning

Learning visual representations through self-supervision is an extremely...
12/03/2021

Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks

Self-supervised learning aims to learn image feature representations wit...
06/14/2021

Self-Supervised Metric Learning in Multi-View Data: A Downstream Task Perspective

Self-supervised metric learning has been a successful approach for learn...
05/03/2019

SCOPS: Self-Supervised Co-Part Segmentation

Parts provide a good intermediate representation of objects that is robu...
12/15/2021

Quantitative analysis of visual representation of sign elements in COVID-19 context

Representation is the way in which human beings re-present the reality o...