Learn to Predict Sets Using Feed-Forward Neural Networks

01/30/2020
by   Hamid Rezatofighi, et al.
30

This paper addresses the task of set prediction using deep feed-forward neural networks. A set is a collection of elements which is invariant under permutation and the size of a set is not fixed in advance. Many real-world problems, such as image tagging and object detection, have outputs that are naturally expressed as sets of entities. This creates a challenge for traditional deep neural networks which naturally deal with structured outputs such as vectors, matrices or tensors. We present a novel approach for learning to predict sets with unknown permutation and cardinality using deep neural networks. In our formulation we define a likelihood for a set distribution represented by a) two discrete distributions defining the set cardinally and permutation variables, and b) a joint distribution over set elements with a fixed cardinality. Depending on the problem under consideration, we define different training models for set prediction using deep neural networks. We demonstrate the validity of our set formulations on relevant vision problems such as: 1)multi-label image classification where we achieve state-of-the-art performance on the PASCAL VOC and MS COCO datasets, 2) object detection, for which our formulation outperforms state-of-the-art detectors such as Faster R-CNN and YOLO v3, and 3) a complex CAPTCHA test, where we observe that, surprisingly, our set-based network acquired the ability of mimicking arithmetics without any rules being coded.

READ FULL TEXT

Authors

page 9

page 10

page 11

page 12

page 14

05/02/2018

Deep Perm-Set Net: Learn to Predict Sets with Unknown Permutation and Cardinality Using Deep Neural Networks

We present a novel approach for learning to predict sets with unknown pe...
11/28/2016

DeepSetNet: Predicting Sets with Deep Neural Networks

This paper addresses the task of set prediction using deep learning. Thi...
09/13/2017

Joint Learning of Set Cardinality and State Distribution

We present a novel approach for learning to predict sets using deep lear...
06/08/2022

Set Interdependence Transformer: Set-to-Sequence Neural Networks for Permutation Learning and Structure Prediction

The task of learning to map an input set onto a permuted sequence of its...
05/31/2018

Interpretable Set Functions

We propose learning flexible but interpretable functions that aggregate ...
04/10/2017

DeepPermNet: Visual Permutation Learning

We present a principled approach to uncover the structure of visual data...
10/20/2019

Sketch2Code: Transformation of Sketches to UI in Real-time Using Deep Neural Network

User Interface (UI) prototyping is a necessary step in the early stages ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.