Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

08/29/2019
by   Ling Liu, et al.
0

Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attractive property of diversity optimized ensemble learning is its robustness against deception: an adversarial perturbation attack can mislead one DNN model to misclassify but may not fool other ensemble DNN members consistently. In this paper we first give an overview of the concept of ensemble diversity and examine the three types of ensemble diversity in the context of DNN classifiers. We then describe a set of ensemble diversity measures, a suite of algorithms for creating diversity ensembles and for performing ensemble consensus (voted or learned) for generating high accuracy ensemble output by strategically combining outputs of individual members. This paper concludes with a discussion on a set of open issues in quantifying ensemble diversity for robust deep learning.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

09/29/2021

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Ensemble learning is a process by which multiple base learners are strat...
10/20/2020

Promoting High Diversity Ensemble Learning with EnsembleBench

Ensemble learning is gaining renewed interests in recent years. This pap...
01/14/2021

DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation

Deep ensembles perform better than a single network thanks to the divers...
06/24/2020

Reducing Overestimation Bias by Increasing Representation Dissimilarity in Ensemble Based Deep Q-Learning

The first deep RL algorithm, DQN, was limited by the overestimation bias...
11/13/2020

Wisdom of the Ensemble: Improving Consistency of Deep Learning Models

Deep learning classifiers are assisting humans in making decisions and h...
09/18/2020

Generating Efficient DNN-Ensembles with Evolutionary Computation

In this work, we leverage ensemble learning as a tool for the creation o...
03/24/2016

Deep Extreme Feature Extraction: New MVA Method for Searching Particles in High Energy Physics

In this paper, we present Deep Extreme Feature Extraction (DEFE), a new ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.