Corner case data description and detection

01/07/2021
by   Tinghui Ouyang, et al.
0

As the major factors affecting the safety of deep learning models, corner cases and related detection are crucial in AI quality assurance for constructing safety- and security-critical systems. The generic corner case researches involve two interesting topics. One is to enhance DL models robustness to corner case data via the adjustment on parameters/structure. The other is to generate new corner cases for model retraining and improvement. However, the complex architecture and the huge amount of parameters make the robust adjustment of DL models not easy, meanwhile it is not possible to generate all real-world corner cases for DL training. Therefore, this paper proposes to a simple and novel study aiming at corner case data detection via a specific metric. This metric is developed on surprise adequacy (SA) which has advantages on capture data behaviors. Furthermore, targeting at characteristics of corner case data, three modifications on distanced-based SA are developed for classification applications in this paper. Consequently, through the experiment analysis on MNIST data and industrial data, the feasibility and usefulness of the proposed method on corner case data detection are verified.

READ FULL TEXT
research
09/06/2019

Data Sanity Check for Deep Learning Systems via Learnt Assertions

Deep learning (DL) techniques have demonstrated satisfactory performance...
research
03/20/2018

DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems

Deep learning defines a new data-driven programming paradigm that constr...
research
03/20/2018

DeepGauge: Comprehensive and Multi-Granularity Testing Criteria for Gauging the Robustness of Deep Learning Systems

Deep learning defines a new data-driven programming paradigm that constr...
research
09/23/2021

DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications

Unsupervised Deep Learning (DL) techniques have been widely used in vari...
research
10/15/2022

Providing Error Detection for Deep Learning Image Classifiers Using Self-Explainability

This paper proposes a self-explainable Deep Learning (SE-DL) system for ...
research
09/15/2020

CorDEL: A Contrastive Deep Learning Approach for Entity Linkage

Entity linkage (EL) is a critical problem in data cleaning and integrati...
research
04/05/2022

Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectors

Robustness of machine learning models on ever-changing real-world data i...

Please sign up or login with your details

Forgot password? Click here to reset