ACETest: Automated Constraint Extraction for Testing Deep Learning Operators

05/29/2023
by   Jingyi Shi, et al.
0

Deep learning (DL) applications are prevalent nowadays as they can help with multiple tasks. DL libraries are essential for building DL applications. Furthermore, DL operators are the important building blocks of the DL libraries, that compute the multi-dimensional data (tensors). Therefore, bugs in DL operators can have great impacts. Testing is a practical approach for detecting bugs in DL operators. In order to test DL operators effectively, it is essential that the test cases pass the input validity check and are able to reach the core function logic of the operators. Hence, extracting the input validation constraints is required for generating high-quality test cases. Existing techniques rely on either human effort or documentation of DL library APIs to extract the constraints. They cannot extract complex constraints and the extracted constraints may differ from the actual code implementation. To address the challenge, we propose ACETest, a technique to automatically extract input validation constraints from the code to build valid yet diverse test cases which can effectively unveil bugs in the core function logic of DL operators. For this purpose, ACETest can automatically identify the input validation code in DL operators, extract the related constraints and generate test cases according to the constraints. The experimental results on popular DL libraries, TensorFlow and PyTorch, demonstrate that ACETest can extract constraints with higher quality than state-of-the-art (SOTA) techniques. Moreover, ACETest is capable of extracting 96.4 1.95 to 55 times more bugs than SOTA techniques. In total, we have used ACETest to detect 108 previously unknown bugs on TensorFlow and PyTorch, with 87 of them confirmed by the developers. Lastly, five of the bugs were assigned with CVE IDs due to their security impacts.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2021

Leveraging Documentation to Test Deep Learning Library Functions

It is integral to test API functions of widely used deep learning (DL) l...
research
02/04/2023

NeuRI: Diversifying DNN Generation via Inductive Rule Inference

Deep Learning (DL) is prevalently used in various industries to improve ...
research
08/03/2022

HirFuzz: Detecting High-Level Optimization Bugs in DL Compilers via Computational Graph Generation

Deep Learning (DL) compilers are widely adopted to optimize advanced DL ...
research
12/08/2022

SkipFuzz: Active Learning-based Input Selection for Fuzzing Deep Learning Libraries

Many modern software systems are enabled by deep learning libraries such...
research
04/04/2023

Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT

Deep Learning (DL) library bugs affect downstream DL applications, empha...
research
07/11/2023

ConFL: Constraint-guided Fuzzing for Machine Learning Framework

As machine learning gains prominence in various sectors of society for a...
research
07/12/2022

Fuzzing Deep-Learning Libraries via Automated Relational API Inference

A growing body of research has been dedicated to DL model testing. Howev...

Please sign up or login with your details

Forgot password? Click here to reset