NeuRI: Diversifying DNN Generation via Inductive Rule Inference

02/04/2023
by   Jiawei Liu, et al.
0

Deep Learning (DL) is prevalently used in various industries to improve decision-making and automate processes, driven by the ever-evolving DL libraries and compilers. The correctness of DL systems is crucial for trust in DL applications. As such, the recent wave of research has been studying the automated synthesis of test-cases (i.e., DNN models and their inputs) for fuzzing DL systems. However, existing model generators only subsume a limited number of operators, for lacking the ability to pervasively model operator constraints. To address this challenge, we propose NeuRI, a fully automated approach for generating valid and diverse DL models composed of hundreds of types of operators. NeuRI adopts a three-step process: (i) collecting valid and invalid API traces from various sources; (ii) applying inductive program synthesis over the traces to infer the constraints for constructing valid models; and (iii) performing hybrid model generation by incorporating both symbolic and concrete operators concolically. Our evaluation shows that NeuRI improves branch coverage of TensorFlow and PyTorch by 51 state-of-the-art. Within four months, NeuRI finds 87 new bugs for PyTorch and TensorFlow, with 64 already fixed or confirmed, and 8 high-priority bugs labeled by PyTorch, constituting 10 Additionally, open-source developers regard error-inducing models reported by us as "high-quality" and "common in practice".

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/29/2023

ACETest: Automated Constraint Extraction for Testing Deep Learning Operators

Deep learning (DL) applications are prevalent nowadays as they can help ...
research
08/02/2022

MEMO: Coverage-guided Model Generation For Deep Learning Library Testing

Recent deep learning (DL) applications are mostly built on top of DL lib...
research
01/17/2022

Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source

Deep learning (DL) systems can make our life much easier, and thus are g...
research
07/26/2022

Finding Deep-Learning Compilation Bugs with NNSmith

Deep-learning (DL) compilers such as TVM and TensorRT are increasingly u...
research
08/13/2020

Graph-Based Fuzz Testing for Deep Learning Inference Engine

Testing deep learning (DL) systems are increasingly crucial as the incre...
research
07/12/2022

Fuzzing Deep-Learning Libraries via Automated Relational API Inference

A growing body of research has been dedicated to DL model testing. Howev...
research
02/08/2023

Fuzzing Automatic Differentiation in Deep-Learning Libraries

Deep learning (DL) has attracted wide attention and has been widely depl...

Please sign up or login with your details

Forgot password? Click here to reset