CODiT: Conformal Out-of-Distribution Detection in Time-Series Data

07/24/2022
by   Ramneet Kaur, et al.
9

Machine learning models are prone to making incorrect predictions on inputs that are far from the training distribution. This hinders their deployment in safety-critical applications such as autonomous vehicles and healthcare. The detection of a shift from the training distribution of individual datapoints has gained attention. A number of techniques have been proposed for such out-of-distribution (OOD) detection. But in many applications, the inputs to a machine learning model form a temporal sequence. Existing techniques for OOD detection in time-series data either do not exploit temporal relationships in the sequence or do not provide any guarantees on detection. We propose using deviation from the in-distribution temporal equivariance as the non-conformity measure in conformal anomaly detection framework for OOD detection in time-series data.Computing independent predictions from multiple conformal detectors based on the proposed measure and combining these predictions by Fisher's method leads to the proposed detector CODiT with guarantees on false detection in time-series data. We illustrate the efficacy of CODiT by achieving state-of-the-art results on computer vision datasets in autonomous driving. We also show that CODiT can be used for OOD detection in non-vision datasets by performing experiments on the physiological GAIT sensory dataset. Code, data, and trained models are available at https://github.com/kaustubhsridhar/time-series-OOD.

READ FULL TEXT

page 2

page 18

page 19

research
08/16/2016

Conformalized density- and distance-based anomaly detection in time-series data

Anomalies (unusual patterns) in time-series data give essential, and oft...
research
10/05/2022

Feature Importance for Time Series Data: Improving KernelSHAP

Feature importance techniques have enjoyed widespread attention in the e...
research
02/21/2023

Using Semantic Information for Defining and Detecting OOD Inputs

As machine learning models continue to achieve impressive performance ac...
research
07/15/2023

Learning Subjective Time-Series Data via Utopia Label Distribution Approximation

Subjective time-series regression (STR) tasks have gained increasing att...
research
05/05/2023

Data Encoding For Healthcare Data Democratisation and Information Leakage Prevention

The lack of data democratization and information leakage from trained mo...
research
03/18/2022

WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series Tasks

Machine learning models often fail to generalize well under distribution...
research
10/12/2021

Real-time Drift Detection on Time-series Data

Practical machine learning applications involving time series data, such...

Please sign up or login with your details

Forgot password? Click here to reset