Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles

06/24/2016
by   Stefan Lee, et al.
0

Many practical perception systems exist within larger processes that include interactions with users or additional components capable of evaluating the quality of predicted solutions. In these contexts, it is beneficial to provide these oracle mechanisms with multiple highly likely hypotheses rather than a single prediction. In this work, we pose the task of producing multiple outputs as a learning problem over an ensemble of deep networks -- introducing a novel stochastic gradient descent based approach to minimize the loss with respect to an oracle. Our method is simple to implement, agnostic to both architecture and loss function, and parameter-free. Our approach achieves lower oracle error compared to existing methods on a wide range of tasks and deep architectures. We also show qualitatively that the diverse solutions produced often provide interpretable representations of task ambiguity.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 6

page 7

11/04/2020

Gradient-Based Empirical Risk Minimization using Local Polynomial Regression

In this paper, we consider the problem of empirical risk minimization (E...
11/19/2015

Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks

Convolutional Neural Networks have achieved state-of-the-art performance...
06/12/2017

Confident Multiple Choice Learning

Ensemble methods are arguably the most trustworthy techniques for boosti...
12/07/2021

On the Effectiveness of Mode Exploration in Bayesian Model Averaging for Neural Networks

Multiple techniques for producing calibrated predictive probabilities us...
02/14/2018

L4: Practical loss-based stepsize adaptation for deep learning

We propose a stepsize adaptation scheme for stochastic gradient descent....
07/22/2019

Speeding Up Iterative Closest Point Using Stochastic Gradient Descent

Sensors producing 3D point clouds such as 3D laser scanners and RGB-D ca...
09/14/2020

The Shooting Regressor; Randomized Gradient-Based Ensembles

An ensemble method is introduced that utilizes randomization and loss fu...

Code Repositories

GREP

GRoup Emotion Parser: A random recurrent deep ensemble based feature extraction framework.


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.