Distribution Awareness for AI System Testing

05/06/2021
by   David Berend, et al.
0

As Deep Learning (DL) is continuously adopted in many safety critical applications, its quality and reliability start to raise concerns. Similar to the traditional software development process, testing the DL software to uncover its defects at an early stage is an effective way to reduce risks after deployment. Although recent progress has been made in designing novel testing techniques for DL software, the distribution of generated test data is not taken into consideration. It is therefore hard to judge whether the identified errors are indeed meaningful errors to the DL application. Therefore, we propose a new OOD-guided testing technique which aims to generate new unseen test cases relevant to the underlying DL system task. Our results show that this technique is able to filter up to 55.44 and is 10.05

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

06/20/2018

Combinatorial Testing for Deep Learning Systems

Deep learning (DL) has achieved remarkable progress over the past decade...
09/05/2019

DeepEvolution: A Search-Based Testing Approach for Deep Neural Networks

The increasing inclusion of Deep Learning (DL) models in safety-critical...
02/24/2022

Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques

Deep Learning (DL) has revolutionized the capabilities of vision-based s...
04/24/2020

Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty

Over the past decade, deep learning (DL) has been successfully applied t...
02/28/2022

Quality Monitoring and Assessment of Deployed Deep Learning Models for Network AIOps

Artificial Intelligence (AI) has recently attracted a lot of attention, ...
05/17/2022

Hierarchical Distribution-Aware Testing of Deep Learning

With its growing use in safety/security-critical applications, Deep Lear...
07/05/2021

DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search

Deep Learning (DL) has been successfully applied to a wide range of appl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.