SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation

05/19/2020
by   Sungmin Kang, et al.
0

The testing of Deep Neural Networks (DNNs) has become increasingly important as DNNs are widely adopted by safety critical systems. While many test adequacy criteria have been suggested, automated test input generation for many types of DNNs remains a challenge because the raw input space is too large to randomly sample or to navigate and search for plausible inputs. Consequently, current testing techniques for DNNs depend on small local perturbations to existing inputs, based on the metamorphic testing principle. We propose new ways to search not over the entire image space, but rather over a plausible input space that resembles the true training distribution. This space is constructed using Variational Autoencoders (VAEs), and navigated through their latent vector space. We show that this space helps efficiently produce test inputs that can reveal information about the robustness of DNNs when dealing with realistic tests, opening the field to meaningful exploration through the space of highly structured images.

READ FULL TEXT

page 3

page 4

page 5

page 6

page 7

research
03/10/2018

Testing Deep Neural Networks

Deep neural networks (DNNs) have a wide range of applications, and softw...
research
02/26/2021

Distribution-Aware Testing of Neural Networks Using Generative Models

The reliability of software that has a Deep Neural Network (DNN) as a co...
research
01/11/2019

Input Prioritization for Testing Neural Networks

Deep neural networks (DNNs) are increasingly being adopted for sensing a...
research
12/11/2020

Automatic Test Suite Generation for Key-points Detection DNNs Using Many-Objective Search

Automatically detecting the positions of key-points (e.g., facial key-po...
research
11/24/2019

DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning

Testing Deep Neural Network (DNN) models has become more important than ...
research
05/16/2022

Prioritizing Corners in OoD Detectors via Symbolic String Manipulation

For safety assurance of deep neural networks (DNNs), out-of-distribution...
research
06/01/2021

Exposing Previously Undetectable Faults in Deep Neural Networks

Existing methods for testing DNNs solve the oracle problem by constraini...

Please sign up or login with your details

Forgot password? Click here to reset