DeepAI AI Chat
Log In Sign Up

Importance-Driven Deep Learning System Testing

02/09/2020
by   Simos Gerasimou, et al.
University of York
Max Planck Institute for Software Systems
Boğaziçi University
0

Deep Learning (DL) systems are key enablers for engineering intelligent applications due to their ability to solve complex tasks such as image recognition and machine translation. Nevertheless, using DL systems in safety- and security-critical applications requires to provide testing evidence for their dependable operation. Recent research in this direction focuses on adapting testing criteria from traditional software engineering as a means of increasing confidence for their correct behaviour. However, they are inadequate in capturing the intrinsic properties exhibited by these systems. We bridge this gap by introducing DeepImportance, a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion for DL systems. Applying IDC enables to establish a layer-wise functional understanding of the importance of DL system components and use this information to assess the semantic diversity of a test set. Our empirical evaluation on several DL systems, across multiple DL datasets and with state-of-the-art adversarial generation techniques demonstrates the usefulness and effectiveness of DeepImportance and its ability to support the engineering of more robust DL systems.

READ FULL TEXT
10/08/2019

Software Engineering Practice in the Development of Deep Learning Applications

Deep-Learning(DL) applications have been widely employed to assist in va...
08/25/2018

Guiding Deep Learning System Testing using Surprise Adequacy

Deep Learning (DL) systems are rapidly being adopted in safety and secur...
02/11/2021

RobOT: Robustness-Oriented Testing for Deep Learning Systems

Recently, there has been a significant growth of interest in applying so...
03/14/2023

Sensitive Region-based Metamorphic Testing Framework using Explainable AI

Deep Learning (DL) is one of the most popular research topics in machine...
07/26/2020

Iterative Boosting Deep Neural Networks for Predicting Click-Through Rate

The click-through rate (CTR) reflects the ratio of clicks on a specific ...
10/10/2018

Secure Deep Learning Engineering: A Software Quality Assurance Perspective

Over the past decades, deep learning (DL) systems have achieved tremendo...