EnsembleNet: End-to-End Optimization of Multi-headed Models

05/24/2019
by   Hanhan Li, et al.
0

Ensembling is a universally useful approach to boost the performance of machine learning models. However, individual models in an ensemble are typically trained independently in separate stages, without information access about the overall ensemble. In this paper, model ensembles are treated as first-class citizens, and their performance is optimized end-to-end with parameter sharing and a novel loss structure that improves generalization. On large-scale datasets including ImageNet, Youtube-8M, and Kinetics, we demonstrate a procedure that starts from a strongly performing single deep neural network, and constructs an EnsembleNet that has both a smaller size and better performance. Moreover, an EnsembleNet can be trained in one stage just like a single model without manual intervention.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/19/2015

Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks

Convolutional Neural Networks have achieved state-of-the-art performance...
research
07/09/2021

Multi-headed Neural Ensemble Search

Ensembles of CNN models trained with different seeds (also known as Deep...
research
04/09/2019

Intra-Ensemble in Neural Networks

Improving model performance is always the key problem in machine learnin...
research
12/06/2018

MEAL: Multi-Model Ensemble via Adversarial Learning

Often the best performing deep neural models are ensembles of multiple b...
research
10/21/2016

End-to-End Training Approaches for Discriminative Segmental Models

Recent work on discriminative segmental models has shown that they can a...
research
01/26/2023

Joint Training of Deep Ensembles Fails Due to Learner Collusion

Ensembles of machine learning models have been well established as a pow...
research
09/05/2017

Sequence Prediction with Neural Segmental Models

Segments that span contiguous parts of inputs, such as phonemes in speec...

Please sign up or login with your details

Forgot password? Click here to reset