Algorithmic Frameworks for the Detection of High Density Anomalies

10/09/2020
by   Ralph Foorthuis, et al.
0

This study explores the concept of high-density anomalies. As opposed to the traditional concept of anomalies as isolated occurrences, high-density anomalies are deviant cases positioned in the most normal regions of the data space. Such anomalies are relevant for various practical use cases, such as misbehavior detection and data quality analysis. Effective methods for identifying them are particularly important when analyzing very large or noisy sets, for which traditional anomaly detection algorithms will return many false positives. In order to be able to identify high-density anomalies, this study introduces several non-parametric algorithmic frameworks for unsupervised detection. These frameworks are able to leverage existing underlying anomaly detection algorithms and offer different solutions for the balancing problem inherent in this detection task. The frameworks are evaluated with both synthetic and real-world datasets, and are compared with existing baseline algorithms for detecting traditional anomalies. The Iterative Partial Push (IPP) framework proves to yield the best detection results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/23/2023

TabADM: Unsupervised Tabular Anomaly Detection with Diffusion Models

Tables are an abundant form of data with use cases across all scientific...
research
04/19/2023

On the Effectiveness of Image Manipulation Detection in the Age of Social Media

Image manipulation detection algorithms designed to identify local anoma...
research
08/16/2020

SECODA: Segmentation- and Combination-Based Detection of Anomalies

This study introduces SECODA, a novel general-purpose unsupervised non-p...
research
05/01/2023

Unsupervised anomaly detection algorithms on real-world data: how many do we need?

In this study we evaluate 32 unsupervised anomaly detection algorithms o...
research
10/04/2022

Multiple Instance Learning for Detecting Anomalies over Sequential Real-World Datasets

Detecting anomalies over real-world datasets remains a challenging task....
research
08/31/2023

Autoencoder-based Online Data Quality Monitoring for the CMS Electromagnetic Calorimeter

The online Data Quality Monitoring system (DQM) of the CMS electromagnet...
research
07/15/2019

Elastic depths for detecting shape anomalies in functional data

We propose a new depth metric called elastic depth that can be used to g...

Please sign up or login with your details

Forgot password? Click here to reset