DeepAI AI Chat
Log In Sign Up

Leveraging Documentation to Test Deep Learning Library Functions

by   Danning Xie, et al.
University of Waterloo
Purdue University

It is integral to test API functions of widely used deep learning (DL) libraries. The effectiveness of such testing requires DL specific input constraints of these API functions. Such constraints enable the generation of valid inputs, i.e., inputs that follow these DL specific constraints, to explore deep to test the core functionality of API functions. Existing fuzzers have no knowledge of such constraints, and existing constraint extraction techniques are ineffective for extracting DL specific input constraints. To fill this gap, we design and implement a document guided fuzzing technique, D2C, for API functions of DL libraries. D2C leverages sequential pattern mining to generate rules for extracting DL specific constraints from API documents and uses these constraints to guide the fuzzing to generate valid inputs automatically. D2C also generates inputs that violate these constraints to test the input validity checking code. In addition, D2C uses the constraints to generate boundary inputs to detect more bugs. Our evaluation of three popular DL libraries (TensorFlow, PyTorch, and MXNet) shows that D2C's accuracy in extracting input constraints is 83.3 baseline fuzzer without input constraints detects only 68 bugs. Most (89) of the 121 bugs are previously unknown, 54 of which have been fixed or confirmed by developers after we report them. In addition, D2C detects 38 inconsistencies within documents, including 28 that are fixed or confirmed after we report them.


page 1

page 2

page 3

page 4


Fuzzing Deep-Learning Libraries via Large Language Models

Detecting bugs in Deep Learning (DL) libraries is critical for almost al...

ACETest: Automated Constraint Extraction for Testing Deep Learning Operators

Deep learning (DL) applications are prevalent nowadays as they can help ...

MEMO: Coverage-guided Model Generation For Deep Learning Library Testing

Recent deep learning (DL) applications are mostly built on top of DL lib...

SkipFuzz: Active Learning-based Input Selection for Fuzzing Deep Learning Libraries

Many modern software systems are enabled by deep learning libraries such...

Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT

Deep Learning (DL) library bugs affect downstream DL applications, empha...

NeuRI: Diversifying DNN Generation via Inductive Rule Inference

Deep Learning (DL) is prevalently used in various industries to improve ...

When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study

Testing Deep Learning (DL) based systems inherently requires large and r...