A Taxonomy of Anomalies in Log Data

11/26/2021
by   Thorsten Wittkopp, et al.
0

Log data anomaly detection is a core component in the area of artificial intelligence for IT operations. However, the large amount of existing methods makes it hard to choose the right approach for a specific system. A better understanding of different kinds of anomalies, and which algorithms are suitable for detecting them, would support researchers and IT operators. Although a common taxonomy for anomalies already exists, it has not yet been applied specifically to log data, pointing out the characteristics and peculiarities in this domain. In this paper, we present a taxonomy for different kinds of log data anomalies and introduce a method for analyzing such anomalies in labeled datasets. We applied our taxonomy to the three common benchmark datasets Thunderbird, Spirit, and BGL, and trained five state-of-the-art unsupervised anomaly detection algorithms to evaluate their performance in detecting different kinds of anomalies. Our results show, that the most common anomaly type is also the easiest to predict. Moreover, deep learning-based approaches outperform data mining-based approaches in all anomaly types, but especially when it comes to detecting contextual anomalies.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

01/09/2018

Precision and Recall for Range-Based Anomaly Detection

Classical anomaly detection is principally concerned with point-based an...
10/27/2021

Sensing Anomalies as Potential Hazards: Datasets and Benchmarks

We consider the problem of detecting, in the visual sensing data stream ...
07/28/2020

Anomaly detection in Context-aware Feature Models

Feature Models are a mechanism to organize the configuration space and f...
07/30/2020

On the Nature and Types of Anomalies: A Review

Anomalies are occurrences in a dataset that are in some way unusual and ...
06/19/2018

CommunityWatch: The Swiss-Army Knife of BGP Anomaly Detection

We present CommunityWatch, an open-source system that enables timely and...
07/15/2019

Elastic depths for detecting shape anomalies in functional data

We propose a new depth metric called elastic depth that can be used to g...
12/18/2018

Anomaly Detection and Interpretation using Multimodal Autoencoder and Sparse Optimization

Automated anomaly detection is essential for managing information and co...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.