DeepPermNet: Visual Permutation Learning

04/10/2017
by   Rodrigo Santa Cruz, et al.
0

We present a principled approach to uncover the structure of visual data by solving a novel deep learning task coined visual permutation learning. The goal of this task is to find the permutation that recovers the structure of data from shuffled versions of it. In the case of natural images, this task boils down to recovering the original image from patches shuffled by an unknown permutation matrix. Unfortunately, permutation matrices are discrete, thereby posing difficulties for gradient-based methods. To this end, we resort to a continuous approximation of these matrices using doubly-stochastic matrices which we generate from standard CNN predictions using Sinkhorn iterations. Unrolling these iterations in a Sinkhorn network layer, we propose DeepPermNet, an end-to-end CNN model for this task. The utility of DeepPermNet is demonstrated on two challenging computer vision problems, namely, (i) relative attributes learning and (ii) self-supervised representation learning. Our results show state-of-the-art performance on the Public Figures and OSR benchmarks for (i) and on the classification and segmentation tasks on the PASCAL VOC dataset for (ii).

READ FULL TEXT

page 7

page 8

research
05/18/2018

Learning Permutations with Sinkhorn Policy Gradient

Many problems at the intersection of combinatorics and computer science ...
research
01/24/2019

AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks

ShuffleNet is a state-of-the-art light weight convolutional neural netwo...
research
12/10/2018

Learning Representations of Sets through Optimized Permutations

Representations of sets are challenging to learn because operations on s...
research
08/25/2023

Kissing to Find a Match: Efficient Low-Rank Permutation Representation

Permutation matrices play a key role in matching and assignment problems...
research
03/26/2020

A Blind Permutation Similarity Algorithm

This paper introduces a polynomial blind algorithm that determines when ...
research
07/25/2022

Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformer

The success of Vision Transformer (ViT) in various computer vision tasks...

Please sign up or login with your details

Forgot password? Click here to reset