-
Understanding Spatial Robustness of Deep Neural Networks
Deep Neural Networks (DNNs) are being deployed in a wide range of settin...
read it
-
FaceHack: Triggering backdoored facial recognition systems using facial characteristics
Recent advances in Machine Learning (ML) have opened up new avenues for ...
read it
-
SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation
The testing of Deep Neural Networks (DNNs) has become increasingly impor...
read it
-
A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding
Over the past few years, deep neural networks (DNNs) have been continuou...
read it
-
Statistical Testing for Efficient Out of Distribution Detection in Deep Neural Networks
Commonly, Deep Neural Networks (DNNs) generalize well on samples drawn f...
read it
-
HateCheck: Functional Tests for Hate Speech Detection Models
Detecting online hate is a difficult task that even state-of-the-art mod...
read it
-
DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
In company with the data explosion over the past decade, deep neural net...
read it
Automatic Test Suite Generation for Key-points Detection DNNs Using Many-Objective Search
Automatically detecting the positions of key-points (e.g., facial key-points or finger key-points) in an image is an essential problem in many applications, such as driver's gaze detection and drowsiness detection in automated driving systems. With the recent advances of Deep Neural Networks (DNNs), Key-Points detection DNNs (KP-DNNs) have been increasingly employed for that purpose. Nevertheless, KP-DNN testing and validation have remained a challenging problem because KP-DNNs predict many independent key-points at the same time – where each individual key-point may be critical in the targeted application – and images can vary a great deal according to many factors. In this paper, we present an approach to automatically generate test data for KP-DNNs using many-objective search. In our experiments, focused on facial key-points detection DNNs developed for an industrial automotive application, we show that our approach can generate test suites to severely mispredict, on average, more than 93 test data generation can only severely mispredict 41 mispredictions, however, are not avoidable and should not therefore be considered failures. We also empirically compare state-of-the-art, many-objective search algorithms and their variants, tailored for test suite generation. Furthermore, we investigate and demonstrate how to learn specific conditions, based on image characteristics (e.g., head posture and skin color), that lead to severe mispredictions. Such conditions serve as a basis for risk analysis or DNN retraining.
READ FULL TEXT
Comments
There are no comments yet.