Interpretable Anomaly Detection with DIFFI: Depth-based Feature Importance for the Isolation Forest

07/21/2020
by   Mattia Carletti, et al.
0

Anomaly Detection is one of the most important tasks in unsupervised learning as it aims at detecting anomalous behaviours w.r.t. historical data; in particular, multivariate Anomaly Detection has an important role in many applications thanks to the capability of summarizing the status of a complex system or observed phenomenon with a single indicator (typically called `Anomaly Score') and thanks to the unsupervised nature of the task that does not require human tagging. The Isolation Forest is one of the most commonly adopted algorithms in the field of Anomaly Detection, due to its proven effectiveness and low computational complexity. A major problem affecting Isolation Forest is represented by the lack of interpretability, as it is not possible to grasp the logic behind the model predictions. In this paper we propose effective, yet computationally inexpensive, methods to define feature importance scores at both global and local level for the Isolation Forest. Moreover, we define a procedure to perform unsupervised feature selection for Anomaly Detection problems based on our interpretability method. We provide an extensive analysis of the proposed approaches, including comparisons against state-of-the-art interpretability techniques. We assess the performance on several synthetic and real-world datasets and make the code publicly available to enhance reproducibility and foster research in the field.

READ FULL TEXT

page 1

page 8

page 9

page 10

research
12/13/2021

Why Are You Weird? Infusing Interpretability in Isolation Forest for Anomaly Detection

Anomaly detection is concerned with identifying examples in a dataset th...
research
11/30/2021

TiWS-iForest: Isolation Forest in Weakly Supervised and Tiny ML scenarios

Unsupervised anomaly detection tackles the problem of finding anomalies ...
research
05/01/2023

Unsupervised anomaly detection algorithms on real-world data: how many do we need?

In this study we evaluate 32 unsupervised anomaly detection algorithms o...
research
06/22/2023

OptIForest: Optimal Isolation Forest for Anomaly Detection

Anomaly detection plays an increasingly important role in various fields...
research
07/08/2022

Active Learning-based Isolation Forest (ALIF): Enhancing Anomaly Detection in Decision Support Systems

The detection of anomalous behaviours is an emerging need in many applic...
research
07/12/2020

Interpretable, Multidimensional, Multimodal Anomaly Detection with Negative Sampling for Detection of Device Failure

Complex devices are connected daily and eagerly generate vast streams of...
research
11/02/2022

DyAnNet: A Scene Dynamicity Guided Self-Trained Video Anomaly Detection Network

Unsupervised approaches for video anomaly detection may not perform as g...

Please sign up or login with your details

Forgot password? Click here to reset