DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

07/05/2021

∙

Deep Learning (DL) has been successfully applied to a wide range of application domains, including safety-critical ones. Several DL testing approaches have been recently proposed in the literature but none of them aims to assess how different interpretable features of the generated inputs affect the system's behaviour. In this paper, we resort to Illumination Search to find the highest-performing test cases (i.e., misbehaving and closest to misbehaving), spread across the cells of a map representing the feature space of the system. We introduce a methodology that guides the users of our approach in the tasks of identifying and quantifying the dimensions of the feature space for a given domain. We developed DeepHyperion, a search-based tool for DL systems that illuminates, i.e., explores at large, the feature space, by providing developers with an interpretable feature map where automatically generated inputs are placed along with information about the exposed behaviours.

READ FULL TEXT

DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks

Feature space exploration as an alternative for design space exploration beyond the parametric space

Data Sanity Check for Deep Learning Systems via Learnt Assertions

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study

Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing

CIDMP: Completely Interpretable Detection of Malaria Parasite in Red Blood Cells using Lower-dimensional Feature Space

An interpretable deep learning method for bearing fault diagnosis

DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

Related Research

DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks

Feature space exploration as an alternative for design space exploration beyond the parametric space

Data Sanity Check for Deep Learning Systems via Learnt Assertions

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study

Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing

CIDMP: Completely Interpretable Detection of Malaria Parasite in Red Blood Cells using Lower-dimensional Feature Space

An interpretable deep learning method for bearing fault diagnosis