ShapeFlow: Dynamic Shape Interpreter for TensorFlow

11/26/2020
by   Sahil Verma, et al.
9

We present ShapeFlow, a dynamic abstract interpreter for TensorFlow which quickly catches tensor shape incompatibility errors, one of the most common bugs in deep learning code. ShapeFlow shares the same APIs as TensorFlow but only captures and emits tensor shapes, its abstract domain. ShapeFlow constructs a custom shape computational graph, similar to the computational graph used by TensorFlow. ShapeFlow requires no code annotation or code modification by the programmer, and therefore is convenient to use. We evaluate ShapeFlow on 52 programs collected by prior empirical studies to show how fast and accurately it can catch shape incompatibility errors compared to TensorFlow. We use two baselines: a worst-case training dataset size and a more realistic dataset size. ShapeFlow detects shape incompatibility errors highly accurately – with no false positives and a single false negative – and highly efficiently – with an average speed-up of 499X and 24X for the first and second baseline, respectively. We believe ShapeFlow is a practical tool that benefits machine learning developers. We will open-source ShapeFlow on GitHub to make it publicly available to both the developer and research communities.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/16/2021

A Static Analyzer for Detecting Tensor Shape Errors in Deep Neural Network Training Code

We present an automatic static analyzer PyTea that detects tensor-shape ...
research
05/03/2019

TensorNetwork on TensorFlow: A Spin Chain Application Using Tree Tensor Networks

TensorNetwork is an open source library for implementing tensor network ...
research
06/05/2023

Security Knowledge-Guided Fuzzing of Deep Learning Libraries

There have been many Deep Learning (DL) fuzzers proposed in the literatu...
research
10/16/2018

AutoGraph: Imperative-style Coding with Graph-based Performance

There is a perceived trade-off between machine learning code that is eas...
research
03/07/2021

Implementing graph neural networks with TensorFlow-Keras

Graph neural networks are a versatile machine learning architecture that...
research
03/16/2022

Gradual Tensor Shape Checking

Tensor shape mismatch is a common source of bugs in deep learning progra...
research
02/06/2020

Sensitivity Analysis in the Dupire Local Volatility Model with Tensorflow

In a recent paper, we have demonstrated how the affinity between TPUs an...

Please sign up or login with your details

Forgot password? Click here to reset