Finding Deep-Learning Compilation Bugs with NNSmith

07/26/2022
by   Jiawei Liu, et al.
0

Deep-learning (DL) compilers such as TVM and TensorRT are increasingly used to optimize deep neural network (DNN) models to meet performance, resource utilization and other requirements. Bugs in these compilers can produce optimized models whose semantics differ from the original models, and produce incorrect results impacting the correctness of down stream applications. However, finding bugs in these compilers is challenging due to their complexity. In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers. Our core approach uses (i) light-weight operator specifications to generate diverse yet valid DNN models allowing us to exercise a large part of the compiler's transformation logic; (ii) a gradient-based search process for finding model inputs that avoid any floating-point exceptional values during model execution, reducing the chance of missed bugs or false alarms; and (iii) differential testing to identify bugs. We implemented this approach in NNSmith which has found 65 new bugs in the last seven months for TVM, TensorRT, ONNXRuntime, and PyTorch. Of these 52 have been confirmed and 44 have been fixed by project maintainers.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/25/2021

NSan: A Floating-Point Numerical Sanitizer

Sanitizers are a relatively recent trend in software engineering. They a...
research
04/19/2022

Muffin: Testing Deep Learning Libraries via Neural Architecture Fuzzing

Deep learning (DL) techniques are proven effective in many challenging t...
research
02/04/2023

NeuRI: Diversifying DNN Generation via Inductive Rule Inference

Deep Learning (DL) is prevalently used in various industries to improve ...
research
02/02/2023

Effective Random Test Generation for Deep Learning Compilers

Deep learning compilers help address difficulties of deploying deep lear...
research
05/04/2022

DEAR: A Novel Deep Learning-based Approach for Automated Program Repair

The existing deep learning (DL)-based automated program repair (APR) mod...
research
04/19/2020

On the Unusual Effectiveness of Type-aware Mutations for Testing SMT Solvers

We propose type-aware operator mutation, a simple, but unusually effecti...
research
06/14/2023

A statistical approach for finding property-access errors

We study the problem of finding incorrect property accesses in JavaScrip...

Please sign up or login with your details

Forgot password? Click here to reset