Persistent Spread Measurement for Big Network Data Based on Register Intersection

04/12/2017
by   You Zhou, et al.
0

Persistent spread measurement is to count the number of distinct elements that persist in each network flow for predefined time periods. It has many practical applications, including detecting long-term stealthy network activities in the background of normal-user activities, such as stealthy DDoS attack, stealthy network scan, or faked network trend, which cannot be detected by traditional flow cardinality measurement. With big network data, one challenge is to measure the persistent spreads of a massive number of flows without incurring too much memory overhead as such measurement may be performed at the line speed by network processors with fast but small on-chip memory. We propose a highly compact Virtual Intersection HyperLogLog (VI-HLL) architecture for this purpose. It achieves far better memory efficiency than the best prior work of V-Bitmap, and in the meantime drastically extends the measurement range. Theoretical analysis and extensive experiments demonstrate that VI-HLL provides good measurement accuracy even in very tight memory space of less than 1 bit per flow.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/30/2018

Per-Flow Cardinality Estimation Based On Virtual LogLog Sketching

Flow cardinality estimation is the problem of estimating the number of d...
research
01/02/2023

ChameleMon: Shifting Measurement Attention as Network State Changes

Flow-level network measurement is critical to many network applications....
research
11/04/2021

Count-Less: A Counting Sketch for the Data Plane of High Speed Switches

Demands are increasing to measure per-flow statistics in the data plane ...
research
12/07/2018

PriMe: Per-Flow Network Measurement by Combining SRAM with DRAM

Network measurement is necessary to obtain an understanding of the netwo...
research
02/11/2019

Scaling Up Anomaly Detection Using In-DRAM Working Set of Active Flows Table

In the zettabyte era, per-flow measurement becomes more challenging owin...
research
12/05/2018

HashFlow For Better Flow Record Collection

Collecting flow records is a common practice of network operators and re...
research
02/02/2019

Phoenix: An Epidemic Approach to Time Reconstruction

Harsh deployment environments and uncertain run-time conditions create n...

Please sign up or login with your details

Forgot password? Click here to reset