Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet

07/01/2020
by   Hao Chen, et al.
20

Ensemble learning is a general technique to improve accuracy in machine learning. However, the heavy computation of a ConvNets ensemble limits its usage in deep learning. In this paper, we present Group Ensemble Network (GENet), an architecture incorporating an ensemble of ConvNets in a single ConvNet. Through a shared-base and multi-head structure, GENet is divided into several groups to make explicit ensemble learning possible in a single ConvNet. Owing to group convolution and the shared-base, GENet can fully leverage the advantage of explicit ensemble learning while retaining the same computation as a single ConvNet. Additionally, we present Group Averaging, Group Wagging and Group Boosting as three different strategies to aggregate these ensemble members. Finally, GENet outperforms larger single networks, standard ensembles of smaller networks, and other recent state-of-the-art methods on CIFAR and ImageNet. Specifically, group ensemble reduces the top-1 error by 1.83 ResNeXt-50 on ImageNet. We also demonstrate its effectiveness on action recognition and object detection tasks.

READ FULL TEXT
research
05/24/2022

Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model

Ensembling is a popular method used to improve performance as a last res...
research
02/17/2020

BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning

Ensembles, where multiple neural networks are trained individually and t...
research
05/13/2016

Wisdom of Crowds cluster ensemble

The Wisdom of Crowds is a phenomenon described in social science that su...
research
03/14/2019

Improving Neural Architecture Search Image Classifiers via Ensemble Learning

Finding the best neural network architecture requires significant time, ...
research
04/17/2019

3D Object Recognition with Ensemble Learning --- A Study of Point Cloud-Based Deep Learning Models

In this study, we present an analysis of model-based ensemble learning f...
research
10/31/2019

Enhancing Certifiable Robustness via a Deep Model Ensemble

We propose an algorithm to enhance certified robustness of a deep model ...
research
09/18/2020

Generating Efficient DNN-Ensembles with Evolutionary Computation

In this work, we leverage ensemble learning as a tool for the creation o...

Please sign up or login with your details

Forgot password? Click here to reset