DeepAI AI Chat
Log In Sign Up

TFCheck : A TensorFlow Library for Detecting Training Issues in Neural Network Programs

by   Houssem Ben Braiek, et al.

The increasing inclusion of Machine Learning (ML) models in safety critical systems like autonomous cars have led to the development of multiple model-based ML testing techniques. One common denominator of these testing techniques is their assumption that training programs are adequate and bug-free. These techniques only focus on assessing the performance of the constructed model using manually labeled data or automatically generated data. However, their assumptions about the training program are not always true as training programs can contain inconsistencies and bugs. In this paper, we examine training issues in ML programs and propose a catalog of verification routines that can be used to detect the identified issues, automatically. We implemented the routines in a Tensorflow-based library named TFCheck. Using TFCheck, practitioners can detect the aforementioned issues automatically. To assess the effectiveness of TFCheck, we conducted a case study with real-world, mutants, and synthetic training programs. Results show that TFCheck can successfully detect training issues in ML code implementations.


page 1

page 2

page 3

page 4


Testing Feedforward Neural Networks Training Programs

Nowadays, we are witnessing an increasing effort to improve the performa...

Bugs in Machine Learning-based Systems: A Faultload Benchmark

The rapid escalation of applying Machine Learning (ML) in various domain...

TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications

As machine learning (ML) has seen increasing adoption in safety-critical...

Tea: Program Repair Using Neural Network Based on Program Information Attention Matrix

The advance in machine learning (ML)-driven natural language process (NL...

Needle in a Haystack: Detecting Subtle Malicious Edits to Additive Manufacturing G-code Files

Increasing usage of Digital Manufacturing (DM) in safety-critical domain...

Finding Label and Model Errors in Perception Data With Learned Observation Assertions

ML is being deployed in complex, real-world scenarios where errors have ...

Carving out the low surface brightness universe with NoiseChisel

NoiseChisel is a program to detect very low signal-to-noise ratio (S/N) ...