DeepAI AI Chat
Log In Sign Up

DLFuzz: Differential Fuzzing Testing of Deep Learning Systems

by   Jianmin Guo, et al.
Shanghai Jiao Tong University
Tsinghua University

Deep learning (DL) systems are increasingly applied to safety-critical domains such as autonomous driving cars. It is of significant importance to ensure the reliability and robustness of DL systems. Existing testing methodologies always fail to include rare inputs in the testing dataset and exhibit low neuron coverage. In this paper, we propose DLFuzz, the frst differential fuzzing testing framework to guide DL systems exposing incorrect behaviors. DLFuzz keeps minutely mutating the input to maximize the neuron coverage and the prediction difference between the original input and the mutated input, without manual labeling effort or cross-referencing oracles from other DL systems with the same functionality. We present empirical evaluations on two well-known datasets to demonstrate its efficiency. Compared with DeepXplore, the state-of-the-art DL whitebox testing framework, DLFuzz does not require extra efforts to find similar functional DL systems for cross-referencing check, but could generate 338.59 with 89.82 coverage, and save 20.11


page 2

page 4


Guiding Deep Learning System Testing using Surprise Adequacy

Deep Learning (DL) systems are rapidly being adopted in safety and secur...

Data Sanity Check for Deep Learning Systems via Learnt Assertions

Deep learning (DL) techniques have demonstrated satisfactory performance...

Automated Testing for Deep Learning Systems with Differential Behavior Criteria

In this work, we conducted a study on building an automated testing syst...

CAGFuzz: Coverage-Guided Adversarial Generative Fuzzing Testing of Deep Learning Systems

Deep Learning systems (DL) based on Deep Neural Networks (DNNs) are more...

Measuring Discrimination to Boost Comparative Testing for Multiple Deep Learning Models

The boom of DL technology leads to massive DL models built and shared, w...

Model-based Exploration of the Frontier of Behaviours for Deep Learning System Testing

With the increasing adoption of Deep Learning (DL) for critical tasks, s...

Testing Deep Learning Models: A First Comparative Study of Multiple Testing Techniques

Deep Learning (DL) has revolutionized the capabilities of vision-based s...