Incorporating Feedback into Tree-based Anomaly Detection

08/30/2017
by   Shubhomoy Das, et al.
0

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomalies are false positives and not interesting from the application perspective. In this paper, we aim to make the analyst's job easier by allowing for analyst feedback during the investigation process. Ideally, the feedback influences the ranking of the anomaly detector in a way that reduces the number of false positives that must be examined before discovering the anomalies of interest. In particular, we introduce a novel technique for incorporating simple binary feedback into tree-based anomaly detectors. We focus on the Isolation Forest algorithm as a representative tree-based anomaly detector, and show that we can significantly improve its performance by incorporating feedback, when compared with the baseline algorithm that does not incorporate feedback. Our technique is simple and scales well as the size of the data increases, which makes it suitable for interactive discovery of anomalies in large datasets.

READ FULL TEXT
research
01/23/2019

Active Anomaly Detection via Ensembles: Insights, Algorithms, and Interpretability

Anomaly detection (AD) task corresponds to identifying the true anomalie...
research
09/15/2020

Improve black-box sequential anomaly detector relevancy with limited user feedback

Anomaly detectors are often designed to catch statistical anomalies. End...
research
09/17/2018

Active Anomaly Detection via Ensembles

In critical applications of anomaly detection including computer securit...
research
11/03/2020

Heartbeat Diagnosis of Performance Anomaly in OpenMP Multi-Threaded Systems

This paper presents a novel heartbeat diagnosis regarding performance an...
research
07/25/2023

Unmasking Anomalies in Road-Scene Segmentation

Anomaly segmentation is a critical task for driving applications, and it...
research
04/19/2023

On the Effectiveness of Image Manipulation Detection in the Age of Social Media

Image manipulation detection algorithms designed to identify local anoma...
research
03/23/2020

Elle: Inferring Isolation Anomalies from Experimental Observations

Users who care about their data store it in databases, which (at least i...

Please sign up or login with your details

Forgot password? Click here to reset