Am I Rare? An Intelligent Summarization Approach for Identifying Hidden Anomalies

12/24/2020
by   Samira Ghodratnama, et al.
0

Monitoring network traffic data to detect any hidden patterns of anomalies is a challenging and time-consuming task that requires high computing resources. To this end, an appropriate summarization technique is of great importance, where it can be a substitute for the original data. However, the summarized data is under the threat of removing anomalies. Therefore, it is vital to create a summary that can reflect the same pattern as the original data. Therefore, in this paper, we propose an INtelligent Summarization approach for IDENTifying hidden anomalies, called INSIDENT. The proposed approach guarantees to keep the original data distribution in summarized data. Our approach is a clustering-based algorithm that dynamically maps original feature space to a new feature space by locally weighting features in each cluster. Therefore, in new feature space, similar samples are closer, and consequently, outliers are more detectable. Besides, selecting representatives based on cluster size keeps the same distribution as the original data in summarized data. INSIDENT can be used both as the preprocess approach before performing anomaly detection algorithms and anomaly detection algorithm. The experimental results on benchmark datasets prove a summary of the data can be a substitute for original data in the anomaly detection task.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/02/2022

DSR – A dual subspace re-projection network for surface anomaly detection

The state-of-the-art in discriminative unsupervised surface anomaly dete...
research
03/13/2019

Crowdsourced wireless spectrum anomaly detection

Automated wireless spectrum monitoring across frequency, time and space ...
research
05/08/2022

Network Traffic Anomaly Detection Method Based on Multi scale Residual Feature

To address the problem that traditional network traffic anomaly detectio...
research
11/26/2021

In-painting Radiography Images for Unsupervised Anomaly Detection

We propose space-aware memory queues for in-painting and detecting anoma...
research
11/19/2021

UN-AVOIDS: Unsupervised and Nonparametric Approach for Visualizing Outliers and Invariant Detection Scoring

The visualization and detection of anomalies (outliers) are of crucial i...
research
11/03/2020

Heartbeat Diagnosis of Performance Anomaly in OpenMP Multi-Threaded Systems

This paper presents a novel heartbeat diagnosis regarding performance an...
research
09/14/2018

Real-Time Nonparametric Anomaly Detection in High-Dimensional Settings

Timely and reliable detection of abrupt anomalies, e.g., faults, intrusi...

Please sign up or login with your details

Forgot password? Click here to reset