Regularizing Deep Networks by Modeling and Predicting Label Structure

04/05/2018
by   Mohammadreza Mostajabi, et al.
0

We construct custom regularization functions for use in supervised training of deep neural networks. Our technique is applicable when the ground-truth labels themselves exhibit internal structure; we derive a regularizer by learning an autoencoder over the set of annotations. Training thereby becomes a two-phase procedure. The first phase models labels with an autoencoder. The second phase trains the actual network of interest by attaching an auxiliary branch that must predict output via a hidden layer of the autoencoder. After training, we discard this auxiliary branch. We experiment in the context of semantic segmentation, demonstrating this regularization strategy leads to consistent accuracy boosts over baselines, both when training from scratch, or in combination with ImageNet pretraining. Gains are also consistent over different choices of convolutional network architecture. As our regularizer is discarded after training, our method has zero cost at test time; the performance improvements are essentially free. We are simply able to learn better network weights by building an abstract model of the label space, and then training the network to understand this abstraction alongside the original task.

READ FULL TEXT

page 2

page 3

page 4

page 7

page 8

research
06/16/2015

Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation

We propose a novel deep neural network architecture for semi-supervised ...
research
10/27/2016

Icon: An Interactive Approach to Train Deep Neural Networks for Segmentation of Neuronal Structures

We present an interactive approach to train a deep neural network pixel ...
research
04/29/2018

SHADE: Information-Based Regularization for Deep Learning

Regularization is a big issue for training deep neural networks. In this...
research
04/29/2018

SHARE: Regularization for Deep Learning

Regularization is a big issue for training deep neural networks. In this...
research
09/09/2019

Self-Teaching Networks

We propose self-teaching networks to improve the generalization capacity...
research
07/07/2020

LabelEnc: A New Intermediate Supervision Method for Object Detection

In this paper we propose a new intermediate supervision method, named La...
research
02/13/2022

A Group-Equivariant Autoencoder for Identifying Spontaneously Broken Symmetries in the Ising Model

We introduce the group-equivariant autoencoder (GE-autoencoder) – a nove...

Please sign up or login with your details

Forgot password? Click here to reset